CI / CD

Configure your CI / CD provider so TesterArmy automatically runs tests when your application is deployed. When a preview deployment completes, TesterArmy receives a webhook, resolves the PR, runs tests against the deployed URL, and posts results as a PR comment.

Coolify

Setup

1. Go to Project → Settings → CI / CD and select Coolify.
2. Click Add Webhook and copy the generated webhook URL.
3. In Coolify, go to Notifications → Webhook.
4. Paste the webhook URL and enable deployment events.

TesterArmy uses pull_request_id from the Coolify payload to identify the PR, so test results are automatically posted as PR comments.

Docker Compose workaround

Coolify has a known issue where Docker Compose deployments don't send the preview URL (preview_fqdn is null). Other build packs (Nixpacks, Dockerfile, Static) work out of the box.

Workaround: When creating or editing the webhook, set a Preview URL pattern:

https://{{pr_number}}.myapp.com

TesterArmy substitutes the PR number from the webhook payload to build the deployment URL.

Vercel

Vercel deployments are handled automatically through the GitHub App integration — no webhook setup needed. TesterArmy detects Vercel preview deployments via GitHub deployment status events and resolves the PR from the commit SHA.

If Vercel deployment protection blocks preview URLs, go to Project → Settings → CI / CD, select Vercel, and add a bypass token. See Project Setup for details.

Troubleshooting

Missing deployment URL

Error: Webhook payload does not contain a deployment URL

• Coolify Docker Compose: Preview URL is null due to a known bug. Configure a Preview URL pattern on the webhook, or switch to Nixpacks/Dockerfile build pack.
• No domain configured: Make sure your application has a domain set in your deployment provider.

No PR comment appears

1. GitHub App connected? Go to Project Settings and verify the GitHub App is installed and the repository is selected.
2. PR number in payload? Coolify sends pull_request_id for preview deployments. For non-preview or production deployments, TesterArmy cannot identify the PR.
3. GitHub App permissions: The app needs Pull requests: Read & Write permissions.

Duplicate events ignored

Each webhook is deduplicated by deployment ID. If you see "reason": "Duplicate event", the same deployment was already processed. Retries from your provider won't create duplicate test runs.

401 Unauthorized

The webhook URL contains the secret token. If you get a 401:

• Verify the URL is correct and hasn't been truncated
• Regenerate the webhook in Project → Settings → CI / CD if the secret was compromised

422 Invalid deployment URL

The deployment URL failed validation:

• URL must be a valid HTTPS URL
• URL must not point to localhost or internal addresses