Module agentforge4j.testkit
Package com.agentforge4j.testkit.assertion

Class WorkflowRunAssert

java.lang.Object
com.agentforge4j.testkit.assertion.WorkflowRunAssert
public final class WorkflowRunAssert extends Object
Fluent, framework-agnostic assertions over a WorkflowRunResult. Every verb is a pure projection over the captured event stream, captured files, and the final state; on a failed expectation a plain AssertionError is thrown, so the engine carries no test-framework dependency. All verbs return this for chaining.

  • Method Details

    • assertThat

      public static WorkflowRunAssert assertThat(WorkflowRunResult result)
      Begins an assertion chain over a run result.
      Parameters:
      result - the run result; must not be null
      Returns:
      a new assertion

    • hasStatus

      public WorkflowRunAssert hasStatus(WorkflowStatus expected)
      Asserts the run's final status.
      Parameters:
      expected - the expected status; must not be null
      Returns:
      this

    • isCompleted

      public WorkflowRunAssert isCompleted()
      Asserts the run completed.
      Returns:
      this

    • isFailed

      public WorkflowRunAssert isFailed()
      Asserts the run failed.
      Returns:
      this

    • failedWith

      public WorkflowRunAssert failedWith(FailureKind kind)
      Asserts the run failed with the given failure kind.
      Parameters:
      kind - the expected failure kind; must not be null
      Returns:
      this

    • failedBecause

      public WorkflowRunAssert failedBecause(String fragment)
      Asserts the run failed and its recorded failure reason contains the given fragment. Use this to pin the specific rejection a negative scenario provokes (a parse error, a path-escape, a fail-closed miss) rather than merely that the run ended FAILED.
      Parameters:
      fragment - expected substring of the failure reason; must not be blank
      Returns:
      this

    • visitedStep

      public WorkflowRunAssert visitedStep(String stepId)
      Asserts the step was visited at least once.
      Parameters:
      stepId - the step id; must not be blank
      Returns:
      this

    • didNotVisitStep

      public WorkflowRunAssert didNotVisitStep(String stepId)
      Asserts the step was never visited.
      Parameters:
      stepId - the step id; must not be blank
      Returns:
      this

    • stepVisitCount

      public WorkflowRunAssert stepVisitCount(String stepId, int expected)
      Asserts the step was visited exactly the expected number of times.
      Parameters:
      stepId - the step id; must not be blank
      expected - the expected visit count
      Returns:
      this

    • stepsInOrderedSubsequence

      public WorkflowRunAssert stepsInOrderedSubsequence(String... stepIds)
      Asserts the given step ids occur, in order, as a subsequence of the visited steps.
      Parameters:
      stepIds - the ordered step ids; must not be null
      Returns:
      this

    • contextHas

      public WorkflowRunAssert contextHas(String key)
      Asserts a context key is present.
      Parameters:
      key - the context key; must not be blank
      Returns:
      this

    • contextMissing

      public WorkflowRunAssert contextMissing(String key)
      Asserts a context key is absent.
      Parameters:
      key - the context key; must not be blank
      Returns:
      this

    • contextNonEmpty

      public WorkflowRunAssert contextNonEmpty(String key)
      Asserts a context key is present and its value is non-empty.
      Parameters:
      key - the context key; must not be blank
      Returns:
      this

    • contextEquals

      public WorkflowRunAssert contextEquals(String key, String expected)
      Asserts a context value equals the expected control value. Reserve this for deterministic control values, never for LLM prose.
      Parameters:
      key - the context key; must not be blank
      expected - the expected value; must not be null
      Returns:
      this

    • contextMatchesRegex

      public WorkflowRunAssert contextMatchesRegex(String key, String regex)
      Asserts a context value matches the given regular expression.
      Parameters:
      key - the context key; must not be blank
      regex - the regular expression; must not be blank
      Returns:
      this

    • contextHasField

      public WorkflowRunAssert contextHasField(String key, String field)
      Asserts a JSON context value has the given top-level field.
      Parameters:
      key - the context key; must not be blank
      field - the JSON field name; must not be blank
      Returns:
      this

    • outputsHaveNoForbiddenTerms

      public WorkflowRunAssert outputsHaveNoForbiddenTerms(Collection<String> forbiddenTerms)
      Asserts that no forbidden term appears in any generated run output — neither in a context value nor in a captured file's content. Matching is case-insensitive and substring-based, so it catches currency codes, money words, and other terms an OSS output must never carry. All violations are reported together.
      Parameters:
      forbiddenTerms - the terms that must be absent; must not be empty
      Returns:
      this

    • createdFile

      public WorkflowRunAssert createdFile(String path)
      Asserts a file was created at the given path.
      Parameters:
      path - the requested path; must not be blank
      Returns:
      this

    • artifactPresent

      public WorkflowRunAssert artifactPresent(String path)
      Alias of createdFile(String).
      Parameters:
      path - the requested path; must not be blank
      Returns:
      this

    • artifactAbsent

      public WorkflowRunAssert artifactAbsent(String path)
      Asserts no file was created at the given path.
      Parameters:
      path - the requested path; must not be blank
      Returns:
      this

    • emittedEvent

      public WorkflowRunAssert emittedEvent(WorkflowEventType type)
      Asserts an event of the given type was emitted.
      Parameters:
      type - the event type; must not be null
      Returns:
      this

    • didNotEmitEvent

      public WorkflowRunAssert didNotEmitEvent(WorkflowEventType type)
      Asserts no event of the given type was emitted.
      Parameters:
      type - the event type; must not be null
      Returns:
      this

    • eventCount

      public WorkflowRunAssert eventCount(WorkflowEventType type, int expected)
      Asserts the exact number of events of the given type.
      Parameters:
      type - the event type; must not be null
      expected - the expected count
      Returns:
      this

    • eventsInOrder

      public WorkflowRunAssert eventsInOrder(WorkflowEventType... types)
      Asserts the given event types occur, in order, as a subsequence of the emitted events.
      Parameters:
      types - the ordered event types; must not be null
      Returns:
      this

    • reachedPendingState

      public WorkflowRunAssert reachedPendingState(WorkflowStatus status)
      Asserts the run is in the given pending state.
      Parameters:
      status - the pending status; must not be null
      Returns:
      this

    • approvalRequested

      public WorkflowRunAssert approvalRequested(String stepId)
      Asserts an approval was requested at the given step.
      Parameters:
      stepId - the step id; must not be blank
      Returns:
      this

    • approvalDecision

      public WorkflowRunAssert approvalDecision(String stepId, ApprovalOutcome outcome)
      Asserts an approval decision with a non-empty reason/note was recorded at the given step.
      Parameters:
      stepId - the step id; must not be blank
      outcome - the expected outcome; must not be null
      Returns:
      this

    • inputRequested

      public WorkflowRunAssert inputRequested(String stepId)
      Asserts input was requested at the given step.
      Parameters:
      stepId - the step id; must not be blank
      Returns:
      this

    • loopIterations

      public WorkflowRunAssert loopIterations(int expected)
      Asserts the iteration count, counted from LOOP_ITERATION_STARTED. This asserts the number of iterations, not the loop strategy — the event stream carries no strategy discriminator, so loop and forEach cannot be told apart here.
      Parameters:
      expected - the expected iteration count
      Returns:
      this

    • forEachIterations

      public WorkflowRunAssert forEachIterations(int expected)
      Asserts the iteration count; an alias of loopIterations(int) over the single canonical LOOP_ITERATION_STARTED source. This asserts the number of iterations, not the loop strategy — the event stream does not distinguish a forEach loop from any other, so the two names imply no distinction the events can back.
      Parameters:
      expected - the expected iteration count
      Returns:
      this

    • invokedTool

      public WorkflowRunAssert invokedTool(String capabilityId)
      Asserts the given tool capability was invoked.
      Parameters:
      capabilityId - the capability id; must not be blank
      Returns:
      this

    • didNotInvokeTool

      public WorkflowRunAssert didNotInvokeTool(String capabilityId)
      Asserts the given tool capability was never invoked.
      Parameters:
      capabilityId - the capability id; must not be blank
      Returns:
      this

    • toolCallCount

      public WorkflowRunAssert toolCallCount(int expected)
      Asserts the number of tool invocations (counted from TOOL_INVOCATION_REQUESTED).
      Parameters:
      expected - the expected count
      Returns:
      this

    • providerCallCount

      public WorkflowRunAssert providerCallCount(int expected)
      Asserts the number of provider (LLM) calls (counted from LLM_CALL_COMPLETED).
      Parameters:
      expected - the expected count
      Returns:
      this

    • providerCallCountForStep

      public WorkflowRunAssert providerCallCountForStep(String stepId, int expected)
      Asserts the number of provider (LLM) calls recorded for a specific step, regardless of dispatch or attempt. Useful for proving a looping or parse-retried step made the expected number of real, metered provider calls — each carries its own stepUid/callAttempt discriminator in its LLM_CALL_COMPLETED payload, but this assertion only needs the step id.
      Parameters:
      stepId - the step id to filter on; must not be blank
      expected - the expected count
      Returns:
      this

    • providerCallTier

      public WorkflowRunAssert providerCallTier(ModelTier tier)
      Asserts at least one provider call resolved its model from the given tier.
      Parameters:
      tier - the expected model tier; must not be null
      Returns:
      this

    • tokenTotals

      public WorkflowRunAssert tokenTotals(int expectedTotal)
      Asserts the deterministic total token usage recorded on the run. Fails when no token total was recorded at all — an absent LLM_TOKENS_TOTAL means token tracking never ran, which must not satisfy an expectation of 0 (a genuinely-tracked zero and a missing tracker are different facts).
      Parameters:
      expectedTotal - the expected total
      Returns:
      this