Skip to main content
A test is the core unit of experimentation in Fleack. It binds a lever to a set of alternative values (variants), targets a specific audience, and tracks a success metric — all without touching your app code or shipping a release. When a test is live, Fleack intercepts eligible API responses and rewrites the targeted parameter on the fly, with each user consistently seeing the same variant across every request.

​
Test status

A test moves through a simple lifecycle:
StatusWhat it means
draftCreated but not yet live. You can edit all settings freely.
runningLive and actively assigning variants to users.
pausedTemporarily stopped. No new assignments; existing data is preserved.
completedEnded. Results are frozen.

​
What a test is made of

Target key — the JSON path of the lever being tested, e.g. data.gems_reward. This is the exact location in the response body that Fleack rewrites for users assigned to a non-control variant. Control value — the value your backend currently returns for that path. Fleack reads this at test-creation time and keeps it fixed for the duration of the test. This ensures the control arm always represents a stable baseline, even if your backend changes the value mid-test. Variants — each variant has an id, a value, and a weight (expressed as a percentage of total traffic). The control automatically receives 100 - sum(variant weights) percent of traffic. For example, if you have two variants each at 33%, the control gets 34%. Segment — which users are eligible. You can target a pre-built segment (e.g. “iOS users with days_since_install ≥ 7”) or leave this set to “All users” to include everyone. See Segments for details. Metrics — what success looks like. You choose one metric per test: a conversion event, a retention day, or a revenue/scalar attribute. See Results for how each metric type is computed.

​
Sticky variant assignment

Once a user is assigned to a variant, they always see that same variant for the life of the test. Assignment is computed by hashing the combination of test_id and user_identifier — so it’s deterministic, requires no stored state, and produces the same result every time the same user makes a request. For requests that can’t be tied to an identified user (no auth header, no user-like path or query parameter), Fleack derives a session-like identity from the user-agent and IP address. This is an approximation — users behind the same NAT may share an identity — but it keeps the exposures counter accurate for anonymous flows.
Variant values are only injected when the targeted JSON path actually exists in the response. If a given request doesn’t include the path (e.g. the backend conditionally omits it), the response passes through unchanged. Fleack never creates new fields from scratch.

​
Launching a test

The fastest path is from the lever catalog: click Test on any lever to open the test creation dialog with the lever pre-selected. Set your variants, allocation, segment, and metric, then click Launch. The test goes live immediately — the next eligible user request will receive a variant. For a full walkthrough, see the Quickstart.

​
Promoting a winner

When the results engine signals that a variant has crossed the winner threshold (≥ 90% win probability for binary metrics, or ≥ 5% uplift for scalar metrics), you can click Promote on that variant. Promotion does two things instantly:
  1. Allocates 100% of traffic to the winning variant.
  2. Marks the test as completed.
No deployment, no engineer, no app store review. The promoted value becomes the default for every user from that moment on.
You don’t have to wait for an automatic winner verdict to promote. If business context makes a clear case for one variant before the threshold is reached, you can promote manually at any time.