# `Aludel.Evals.FileSuite`
[🔗](https://github.com/ccarvalho-eng/aludel/blob/main/lib/aludel/evals/file_suite.ex#L1)

Loads and executes versioned JSON or YAML suite manifests.

A manifest identifies a persisted suite, prompt version, and provider. It may
also define bounded repeated-sampling settings. Test cases and dataset
provenance remain owned by the database; executing a manifest never imports,
replaces, or deletes suite data.

Schema version 1 accepts this shape:

    schema_version: 1
    suite_id: 9a756a58-eaec-43ca-99e6-f5c016d85d0c
    prompt_version_id: e74cf2e1-94b6-4bcb-9ed9-b259661be906
    provider_id: e1c60ec0-6d55-419b-b958-7d088055254f
    sampling:
      samples: 5
      reducer: majority

Supported reducers are `all`, `any`, `majority`, and
`minimum_pass_rate`. The latter also requires `minimum_pass_rate` between
`0.0` and `1.0`.

Files are limited to 256 KiB. Parsing rejects unsupported extensions,
invalid UTF-8, aliases, explicit YAML tags, multiple YAML documents,
duplicate mapping keys, unknown fields, and unsupported schema versions
before database access.

# `error`

```elixir
@type error() :: %{code: String.t(), message: String.t()}
```

# `format`

```elixir
@type format() :: :json | :yaml
```

# `t`

```elixir
@type t() :: %Aludel.Evals.FileSuite{
  prompt_version_id: Ecto.UUID.t(),
  provider_id: Ecto.UUID.t(),
  sampling: Aludel.Evals.Sampling.t(),
  schema_version: 1,
  suite_id: Ecto.UUID.t()
}
```

# `execute`

```elixir
@spec execute(t()) :: {:ok, Aludel.Evals.SuiteRun.t()} | {:error, error()}
```

Executes a validated manifest against its persisted suite.

The prompt version must belong to the suite's prompt. The suite's existing
test cases, document associations, and latest quality policy are used without
modifying dataset ownership.

# `load`

```elixir
@spec load(Path.t()) :: {:ok, t()} | {:error, error()}
```

Loads and validates a `.json`, `.yaml`, or `.yml` manifest.

The file is read with a fixed upper bound. Errors have stable `code` and
`message` fields suitable for command-line and application handling.

# `load_and_execute`

```elixir
@spec load_and_execute(Path.t()) ::
  {:ok, Aludel.Evals.SuiteRun.t()} | {:error, error()}
```

Loads, validates, and executes a suite manifest.

Invalid files and targets return stable errors instead of starting a partial
evaluation.

# `load_string`

```elixir
@spec load_string(binary(), format()) :: {:ok, t()} | {:error, error()}
```

Parses and validates manifest content in `:json` or `:yaml` format.

The same size, encoding, duplicate-key, schema, identifier, and sampling
checks used by `load/1` apply to in-memory content.

---

*Consult [api-reference.md](api-reference.md) for complete listing*