How experiments work

An experiment splits your visitors into variants, measures a goal, and reports which variant wins. Versuch keeps assignment deterministic and sticky so every surface (browser, server, dashboard) agrees on which variant a visitor sees.

The three types

element

The default. Swaps text, HTML, or attributes on a CSS selector. The tracker applies it client-side before paint, so there is no flicker. Use it for headline, copy, and layout tests.

cta

Overrides a button or link (an element override, or a hosted component Versuch renders for you). The default goal is cta_click. Use it for call-to-action wording, colour, and destination tests.

manual

Server- or code-controlled, with no auto-apply. You fetch the variant yourself and render it however you like. Use it for server-rendered pages, pricing logic, onboarding flows, or anything the tracker cannot reach into.

When to use each

• Reach for element first. If you can point at the thing you want to change with a CSS selector and it lives in the DOM, this is the least code.
• Use cta when the change is specifically a button or link, or when you want Versuch to render a hosted CTA rather than override an existing one.
• Use manual when the variant has to be decided outside the browser DOM: server-rendered HTML, a feature branch in your own code, a price shown from your backend. You get the variant via versuch.experiment(key) in the browser or POST /v1/experiments/:key/assign on the server.

Deterministic sticky assignment

A visitor’s variant is a deterministic hash of their visitor id plus the experiment key. There is no server round-trip to decide it and nothing is stored per visitor: the same visitor id and key always hash to the same variant, so a returning visitor stays in the same bucket, and a browser call and a server call agree for the same unit.

visitor id + key inputs

→

hash deterministic

→

variant control / b, sticky

The same inputs always map to the same variant, with no per-visitor storage.

Because assignment hashes on the variant keys and their order, keys are fixed after creation. Changing them would re-bucket everyone.

Exposures vs conversions

Two distinct events drive results:

• Exposure: the visitor was actually put into a variant. For element and cta this fires when the variant is applied to the page. For manual it fires on the first versuch.experiment(key) call or the first assign POST for that unit.
• Conversion: the visitor did the thing you are measuring. You record it with versuch.track("goal") in the browser or a server-side track, tied to the variant the unit was assigned.

Results compare conversions against exposures per variant, so a visitor only counts once they have actually been exposed.

Goals

The goal is the event name that counts as a conversion, for example signup or purchase. For cta experiments the default goal is cta_click. For manual experiments a goal is required at creation: it is the event name you will emit for conversions.

Stats engines

Every experiment is analysed by one of two engines, chosen at creation with statsEngine (default bayesian):

• bayesian (default), reports the probability a variant beats control and a credible uplift range. Reads naturally as “variant b has an 87% chance to win.”
• frequentist: reports a p-value and significance against control, the classic hypothesis-test framing.

Pick one at creation; it decides how results are computed and presented.