QA test automation for sign-up and OTP emails

Sign-up confirmation, password reset, magic-link login, email change, two-factor codes: each is a flow users hit every day and tests often skip, because the email is hard to get at. Agentboxd makes the email step a single call in your test, with a real message delivered over the internet by the same provider your users get mail from.

#The flow

• Before the test, create a temporary inbox (ttl_seconds, 60 s to 24 h). It has a random address on tmp.agentboxd.com, can’t send, and belongs to this test only.
• In the test, record the time, then submit the form with the inbox address.
• Wait with GET /v1/inboxes/:id/verification?since=…&from=…. It returns the newest matching message’s code and link as soon as it arrives, with a confidence score, or null after the timeout.
• Use the code in the form, or open the link.
• After the test, delete the inbox; its messages are wiped at once. If the run dies, it wipes itself at expires_at.

#In pytest

import pytest
from datetime import datetime, timezone
from agentboxd import Agentboxd

mr = Agentboxd()  # AGENTBOXD_API_KEY from the CI secret


@pytest.fixture
def inbox():
    box = mr.inboxes.create_temporary(ttl_seconds=900)
    yield box
    mr.inboxes.delete(box["id"])  # wipes its mail now


def test_password_reset(app, inbox):
    app.sign_up(email=inbox["address"])  # your app client or page object
    since = datetime.now(timezone.utc).isoformat()
    app.request_password_reset(email=inbox["address"])

    v = mr.messages.wait_for_verification(inbox["id"], since=since, from_="acme.example", timeout=30)
    assert v is not None and v["link"], "no reset email within 30 s"
    app.open(v["link"])

The same pattern in Playwright, as a fixture with a CI-ready config: Playwright email verification guide.

#What it catches that a mail stub doesn’t

• The provider really accepted and delivered the message (credentials, sending limits, a changed API).
• The link points to the right host for the environment, and the token in it works.
• Your sending setup authenticates: every received message carries the SPF, DKIM and DMARC results, and failures are labelled spf-fail or dmarc-fail.
• Codes are found by our extractor, which ignores prices, dates, phone and order numbers, so a template change that moves the code doesn’t break your regex, because you have none.

#Limits to plan for

A workspace can have 25 active temporary inboxes and create 60 an hour. Suites bigger than that can create permanent inboxes with a client_id per test (idempotent, reused across runs) and filter each wait with since. Mail received by temporary inboxes counts toward the plan’s monthly emails; the inboxes themselves don’t count toward the inbox limit. See Temporary inboxes.