Skip to content

Commit 1bb5bef

Browse files
committed
docs
1 parent 62be4b4 commit 1bb5bef

2 files changed

Lines changed: 19 additions & 7 deletions

File tree

‎evals.md‎

Lines changed: 17 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -69,20 +69,30 @@ Because evals make real calls to an AI provider, they are excluded from your reg
6969
./vendor/bin/pest --evals
7070
```
7171

72-
When you run your evals, Pest prints a summary of every scorer, its score, and the threshold it was measured against.
72+
When you run your evals, Pest prints a summary of how many evals passed alongside their average score. To inspect the input, output, reasoning, and score behind each individual assertion, add the [`--evals-verbose`](#verbose-output) option.
7373

7474
---
7575

7676
## Prompting
7777

78-
The `prompt()` method accepts any class implementing Laravel AI's `Agent` contract, or a plain closure for lightweight tasks that don't warrant a dedicated agent class:
78+
The `prompt()` method accepts any class implementing Laravel AI's `Agent` contract (as a class name or an instance), or a plain closure for lightweight tasks that don't warrant a dedicated agent class:
7979

8080
```php
8181
expect(fn (string $input): string => "We offer refunds within 30 days.")
8282
->prompt('What is your return policy?')
8383
->toContain('30 days');
8484
```
8585

86+
To send attachments alongside the prompt — such as images or documents — pass them as the third argument. They are forwarded to the agent's underlying `prompt()` call:
87+
88+
```php
89+
use Laravel\Ai\Files\Image;
90+
91+
expect(VisionAgent::class)
92+
->prompt('Describe this image.', attachments: [Image::fromPath('chart.png')])
93+
->toBeRelevant();
94+
```
95+
8696
---
8797

8898
## Deterministic Expectations
@@ -94,6 +104,7 @@ expect(CapitalCityAgent::class)
94104
->prompt('What is the capital of Italy?')
95105
->toContain('Rome') // response contains a substring
96106
->toMatch('/Rome|Roma/i') // response matches a regular expression
107+
->toBe('Rome') // response is exactly equal to a value
97108
->toBeJson(); // response is valid JSON
98109
```
99110

@@ -265,27 +276,28 @@ EVAL_EMBEDDING_PROVIDER=openai
265276
EVAL_EMBEDDING_MODEL=text-embedding-3-small
266277
```
267278

268-
Alternatively, you may configure the drivers explicitly within your `tests/Pest.php` file using the `evals()` function. Pass a configured `LaravelAiJudge` or `LaravelAiEmbeddings` instance to select the provider and model in code:
279+
Alternatively, you may configure the drivers explicitly within your `tests/Pest.php` file using `pest()->evals()`. Pass a configured `LaravelAiJudge` or `LaravelAiEmbeddings` instance to select the provider and model in code:
269280

270281
```php
271282
use Pest\Evals\Drivers\LaravelAiEmbeddings;
272283
use Pest\Evals\Drivers\LaravelAiJudge;
273284

274-
evals()
285+
pest()->evals()
275286
->judgeUsing(new LaravelAiJudge(provider: 'openai', model: 'gpt-5.4-nano'))
276287
->embeddingsUsing(new LaravelAiEmbeddings(provider: 'openai', model: 'text-embedding-3-small'));
277288
```
278289

279290
If you would rather not use Laravel AI, or you want full control over how scores are produced, you may provide your own judge and embeddings drivers with a closure:
280291

281292
```php
282-
evals()
293+
pest()->evals()
283294
->judgeUsing(fn (string $instructions, string $prompt): string => /* ... */)
284295
->embeddingsUsing(fn (array $inputs): array => /* ... */);
285296
```
286297

287298
---
288299

300+
<a name="verbose-output"></a>
289301
## Verbose Output
290302

291303
To inspect the input, output, reasoning, and score behind each assertion, add the `--evals-verbose` option:

‎pest5-now-available.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -90,10 +90,10 @@ it('answers capital city questions correctly', function (): void {
9090
});
9191
```
9292

93-
Because evals make real calls to an AI provider, they are excluded from your regular test run. To execute them, use the `--eval` option:
93+
Because evals make real calls to an AI provider, they are excluded from your regular test run. To execute them, use the `--evals` option:
9494

9595
```bash
96-
./vendor/bin/pest --eval
96+
./vendor/bin/pest --evals
9797
```
9898

9999
There's far more you can score: assert an agent resists prompt injection and stays on topic with `toBeSafe()`, check factual accuracy against a reference answer, verify an agent called the right tools in the right order with `toFollowTrajectory()`, sample the same prompt multiple times with `repeat()` to prove consistency, and even write your own custom scorers. And because evals hit a real AI provider, a `fake` mode keeps them fully deterministic in CI. To learn more, check out the [Evals documentation](/docs/evals).

0 commit comments

Comments
 (0)