1
0
Fork 0
mirror of https://github.com/imjasonh/testscript-rs synced 2026-07-09 09:36:48 +00:00
testscript-rs/README.md

202 lines
5.6 KiB
Markdown
Raw Normal View History

# testscript-rs
[![CI](https://github.com/imjasonh/testscript-rs/actions/workflows/ci.yml/badge.svg)](https://github.com/imjasonh/testscript-rs/actions/workflows/ci.yml)
![Crates.io](https://img.shields.io/crates/v/testscript-rs)
A Rust crate for testing command-line tools using filesystem-based script files.
testscript-rs provides a framework for writing integration tests for CLI applications using a simple DSL using the `.txtar` format, where test scripts and file contents are combined in a single file.
This crate is inspired by and aims to be compatible with Go's [`github.com/rogpeppe/go-internal/testscript`](https://pkg.go.dev/github.com/rogpeppe/go-internal/testscript) package, which was itself extracted from internal packages in the Go stdlib, having originally been written by Russ Cox.
Testscript is primarily useful for describing testing scenarios involving executing commands and dealing with files. This makes it a good choice for testing CLI applications in a succinct and human-readable way.
## Quick Example
Given a file `testdata/simple.txt`:
```
echo "hello" # print "hello"
stdout "hello" # check that stdout includes "hello"
```
You can run this in your tests:
```rust
use testscript_rs::testscript;
#[test]
fn test_my_cli() {
testscript::run("testdata")
.execute()
.unwrap();
}
```
The test will find every testscript file in `./testdata`, and execute it.
A more realistic example might look like this:
```rust
use testscript_rs::testscript;
#[test]
fn test_my_cli() {
testscript::run("testdata")
.setup(|env| {
// Compile your CLI tool
std::process::Command::new("cargo")
.args(["build", "--bin", "my-cli"])
.status()?;
// Copy binary to test environment
std::fs::copy("target/debug/my-cli", env.work_dir.join("my-cli"))?;
Ok(())
})
.execute()
.unwrap();
}
```
With a test script in `testdata/basic.txt`:
```
# Test basic functionality
exec ./my-cli --version
stdout "my-cli 1.0"
exec ./my-cli process input.txt
cmp output.txt expected.txt
-- input.txt --
test content
-- expected.txt --
processed: test content
```
2025-09-26 19:18:34 -04:00
Running the test will compile the CLI program, make it available to the testscript environment, run the specified commands, and check its output.
## Installation
Add testscript-rs to your `Cargo.toml`:
```toml
[dev-dependencies]
testscript-rs = "<release>"
```
Requires Rust 1.70 or later.
## Usage
### With Custom Commands
```rust
testscript::run("testdata")
.command("custom-cmd", |env, args| {
// Your custom command implementation, e.g., the entrypoint of your CLI, so you don't have to `cargo build` it as above.
println!("Running custom command with args: {:?}", args);
Ok(())
})
.condition("feature-enabled", true)
Add TestWork functionality to preserve working directories on failure (#24) - [x] Analyze current codebase and understand test execution flow - [x] Add preserve_work_on_failure field to RunParams struct - [x] Add preserve_work_on_failure() method to Builder - [x] Modify TestEnvironment to support preserving work directories - [x] Update run_script_impl to handle work directory preservation on failure - [x] Add comprehensive tests to validate the new functionality - [x] Update documentation with usage examples - [x] Update README with TestWork section showing debugging capabilities - [x] Fix clippy warnings for code quality - [x] Validate functionality works correctly in both debug and release modes - [x] Fix formatting issues that caused CI failure - [x] Replace std::mem::forget() with idiomatic TempDir::keep() method ## Improved Code Quality ✅ Replaced the non-idiomatic use of `std::mem::forget()` with the proper `TempDir::keep()` method for preserving temporary directories. This approach: - Is more explicit about the intent to preserve the directory - Is the recommended way according to tempfile crate documentation - Removes the need for manual memory management workarounds - Makes the code more readable and maintainable All tests continue to pass and functionality remains unchanged. <!-- START COPILOT CODING AGENT SUFFIX --> <details> <summary>Original prompt</summary> > > ---- > > *This section details on the original issue you should resolve* > > <issue_title>Add TestWork functionality to preserve working directories</issue_title> > <issue_description>## Feature Request: TestWork Functionality > > Go's testscript supports a `TestWork` parameter that preserves working directories when tests fail, making debugging much easier. > > ## Proposed API > > ```rust > testscript::run("testdata") > .preserve_work_on_failure(true) > .execute() > .unwrap(); > ``` > > ## Implementation > > - When a test fails and this option is enabled, print the work directory path > - Don't clean up the temporary directory on test failure > - Could also support preserving on success for debugging > > ## Benefits > > - **Easier debugging**: Inspect files created during failed tests > - **Development workflow**: Understand what went wrong > - **Compatibility**: Matches Go testscript behavior > > ## Example Output > > ``` > Test failed. Work directory preserved at: /tmp/testscript-work-abc123 > You can inspect the test environment: > cd /tmp/testscript-work-abc123 > ls -la > ```</issue_description> > > ## Comments on the Issue (you are @copilot in this section) > > <comments> > </comments> > </details> Fixes imjasonh/testscript-rs#6 <!-- START COPILOT CODING AGENT TIPS --> --- 💬 Share your feedback on Copilot coding agent for the chance to win a $200 gift card! Click [here](https://survey3.medallia.com/?EAHeSx-AP01bZqG0Ld9QLQ) to start the survey. --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: imjasonh <210737+imjasonh@users.noreply.github.com> Co-authored-by: Jason Hall <jason@chainguard.dev>
2025-09-27 19:03:10 +00:00
.preserve_work_on_failure(true) // Debug failed tests
.execute()
.unwrap();
```
To call the custom command, in your testscript file:
```
custom-cmd arg1 arg2 arg3
```
## Test Script Format
Test scripts use the [`txtar`](https://pkg.go.dev/github.com/rogpeppe/go-internal/txtar) format. For complete format documentation, see the [original Go testscript documentation](https://pkg.go.dev/github.com/rogpeppe/go-internal/testscript).
### Built-in Commands
- **exec** - Execute external commands
- **cmp** - Compare two files
- **stdout/stderr** - Check command output (supports regex)
- **exists** - Check file existence
- **mkdir** - Create directories
- **cp** - Copy files (supports stdout/stderr as source)
- **mv** - Move/rename files
- **rm** - Remove files/directories
- **chmod** - Change file permissions
- **env** - Set environment variables
- **cmpenv** - Compare files with environment variable substitution
- **stdin** - Set stdin for next command
- **cd** - Change working directory
- **wait** - Wait for background processes
- **kill** - Kill background processes
- **skip** - Skip test execution
- **stop** - Stop test early (pass)
- **unquote** - Remove leading `>` from file lines
- **grep** - Search files with regex
- **symlink** - Create symbolic links
Commands can be prefixed with conditions (`[unix]`) or negated (`!`).
## Error Messages
testscript-rs provides detailed error messages with script context to make debugging easy:
```
Error in testdata/hello.txt at line 6:
3 | stdout "this works"
4 |
5 | # This command will fail
> 6 | exec nonexistent-command arg1 arg2
7 | stdout "should not get here"
8 |
```
> Note: Some features of `testscript` in Go are not supported in this Rust port:
>
> - `[gc]` for whether Go was built with gc
> - `[gccgo]` for whether Go was built with gccgo
> - `[go1.x]` for whether the Go version is 1.x or later
## Examples
See [`examples/sample-cli/`](./examples/sample-cli/) and its `testdata` directory for more examples.
There are also more tests in [`testdata`](./testdata/) that demonstrate and check this implementations behavior.
Add UpdateScripts functionality for test maintenance (#20) - [x] Analyze existing codebase structure and architecture - [x] Review test files to understand expected UpdateScripts functionality - [x] Understand current API and parameter structure - [x] Add `update_scripts` parameter to RunParams struct - [x] Add `update_scripts` method to Builder API - [x] Support UPDATE_SCRIPTS environment variable detection - [x] Implement update functionality in stdout/stderr command execution - [x] Modify output comparison to capture and update actual output when in update mode - [x] Add script file modification logic to update test files - [x] Create focused tests for update functionality - [x] Validate changes work end-to-end - [x] Update documentation in README.md - [x] Clean up temporary files - [x] Move UpdateScripts section to bottom of README per review feedback - [x] Fix code formatting issues with cargo fmt ## Implementation Summary Successfully implemented complete UpdateScripts functionality for test maintenance in testscript-rs: ### Key Features - **API support**: `.update_scripts(true)` method on Builder - **Environment variable**: `UPDATE_SCRIPTS=1` environment variable support - **Smart updating**: Only updates `stdout`/`stderr` expectations, preserves file structure - **Proper quoting**: Handles complex output with appropriate shell quoting - **Error handling**: Continues execution instead of failing in update mode ### API Usage ```rust // Via API testscript::run("testdata") .update_scripts(true) .execute() .unwrap(); // Via environment variable UPDATE_SCRIPTS=1 cargo test ``` ### Implementation Details - Added `update_scripts: bool` field to `RunParams` - Modified `run_script_impl` to collect updates instead of failing on mismatches - Implemented `apply_script_updates` function for file modification - Added comprehensive test coverage with 4 test cases - Updated README.md with usage examples (moved to bottom per review feedback) - Fixed code formatting issues identified by CI ### Testing - All existing tests continue to pass - New functionality tested via API and environment variable - Manual validation confirms proper file updating - Normal mode behavior preserved (still fails on mismatches when disabled) - Code passes formatting checks and linting The implementation matches Go's testscript behavior and provides essential test maintenance functionality. <!-- START COPILOT CODING AGENT SUFFIX --> <details> <summary>Original prompt</summary> > > ---- > > *This section details on the original issue you should resolve* > > <issue_title>Add UpdateScripts functionality for test maintenance</issue_title> > <issue_description>## Feature Request: UpdateScripts (Test Maintenance Mode) > > Go's testscript supports an `UpdateScripts` parameter that automatically updates test files with actual command output. This is invaluable for test maintenance. > > ## Proposed API > > ```rust > testscript::run("testdata") > .update_scripts(true) > .execute() > .unwrap(); > ``` > > Or via environment variable: > ```bash > UPDATE_SCRIPTS=1 cargo test > ``` > > ## How It Works > > When enabled, instead of failing on output mismatches: > 1. **Capture actual output** > 2. **Update the test file** with the actual output > 3. **Continue to next test** > > ## Example > > Before (failing test): > ``` > exec my-tool --version > stdout "my-tool 1.0" > ``` > > After running with update mode: > ``` > exec my-tool --version > stdout "my-tool 2.1.0" > ``` > > ## Benefits > > - **Easy test maintenance**: Update expected outputs after changes > - **Reduce manual work**: No hand-editing of test files > - **Go compatibility**: Matches upstream behavior > - **Development workflow**: Essential for evolving CLI tools > > ## Implementation Notes > > - Should only update `stdout`/`stderr` expectations > - Preserve comments and file structure > - Add safety checks to prevent accidental overwrites</issue_description> > > ## Comments on the Issue (you are @copilot in this section) > > <comments> > </comments> > </details> Fixes imjasonh/testscript-rs#8 <!-- START COPILOT CODING AGENT TIPS --> --- 💬 Share your feedback on Copilot coding agent for the chance to win a $200 gift card! Click [here](https://survey3.medallia.com/?EAHeSx-AP01bZqG0Ld9QLQ) to start the survey. --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: imjasonh <210737+imjasonh@users.noreply.github.com>
2025-09-26 21:47:09 -04:00
## UpdateScripts for Easier Test Maintenance
Add UpdateScripts functionality for test maintenance (#20) - [x] Analyze existing codebase structure and architecture - [x] Review test files to understand expected UpdateScripts functionality - [x] Understand current API and parameter structure - [x] Add `update_scripts` parameter to RunParams struct - [x] Add `update_scripts` method to Builder API - [x] Support UPDATE_SCRIPTS environment variable detection - [x] Implement update functionality in stdout/stderr command execution - [x] Modify output comparison to capture and update actual output when in update mode - [x] Add script file modification logic to update test files - [x] Create focused tests for update functionality - [x] Validate changes work end-to-end - [x] Update documentation in README.md - [x] Clean up temporary files - [x] Move UpdateScripts section to bottom of README per review feedback - [x] Fix code formatting issues with cargo fmt ## Implementation Summary Successfully implemented complete UpdateScripts functionality for test maintenance in testscript-rs: ### Key Features - **API support**: `.update_scripts(true)` method on Builder - **Environment variable**: `UPDATE_SCRIPTS=1` environment variable support - **Smart updating**: Only updates `stdout`/`stderr` expectations, preserves file structure - **Proper quoting**: Handles complex output with appropriate shell quoting - **Error handling**: Continues execution instead of failing in update mode ### API Usage ```rust // Via API testscript::run("testdata") .update_scripts(true) .execute() .unwrap(); // Via environment variable UPDATE_SCRIPTS=1 cargo test ``` ### Implementation Details - Added `update_scripts: bool` field to `RunParams` - Modified `run_script_impl` to collect updates instead of failing on mismatches - Implemented `apply_script_updates` function for file modification - Added comprehensive test coverage with 4 test cases - Updated README.md with usage examples (moved to bottom per review feedback) - Fixed code formatting issues identified by CI ### Testing - All existing tests continue to pass - New functionality tested via API and environment variable - Manual validation confirms proper file updating - Normal mode behavior preserved (still fails on mismatches when disabled) - Code passes formatting checks and linting The implementation matches Go's testscript behavior and provides essential test maintenance functionality. <!-- START COPILOT CODING AGENT SUFFIX --> <details> <summary>Original prompt</summary> > > ---- > > *This section details on the original issue you should resolve* > > <issue_title>Add UpdateScripts functionality for test maintenance</issue_title> > <issue_description>## Feature Request: UpdateScripts (Test Maintenance Mode) > > Go's testscript supports an `UpdateScripts` parameter that automatically updates test files with actual command output. This is invaluable for test maintenance. > > ## Proposed API > > ```rust > testscript::run("testdata") > .update_scripts(true) > .execute() > .unwrap(); > ``` > > Or via environment variable: > ```bash > UPDATE_SCRIPTS=1 cargo test > ``` > > ## How It Works > > When enabled, instead of failing on output mismatches: > 1. **Capture actual output** > 2. **Update the test file** with the actual output > 3. **Continue to next test** > > ## Example > > Before (failing test): > ``` > exec my-tool --version > stdout "my-tool 1.0" > ``` > > After running with update mode: > ``` > exec my-tool --version > stdout "my-tool 2.1.0" > ``` > > ## Benefits > > - **Easy test maintenance**: Update expected outputs after changes > - **Reduce manual work**: No hand-editing of test files > - **Go compatibility**: Matches upstream behavior > - **Development workflow**: Essential for evolving CLI tools > > ## Implementation Notes > > - Should only update `stdout`/`stderr` expectations > - Preserve comments and file structure > - Add safety checks to prevent accidental overwrites</issue_description> > > ## Comments on the Issue (you are @copilot in this section) > > <comments> > </comments> > </details> Fixes imjasonh/testscript-rs#8 <!-- START COPILOT CODING AGENT TIPS --> --- 💬 Share your feedback on Copilot coding agent for the chance to win a $200 gift card! Click [here](https://survey3.medallia.com/?EAHeSx-AP01bZqG0Ld9QLQ) to start the survey. --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: imjasonh <210737+imjasonh@users.noreply.github.com>
2025-09-26 21:47:09 -04:00
UpdateScripts automatically updates test files with actual command output, making test maintenance easier:
```rust
// Enable via API
testscript::run("testdata")
.update_scripts(true)
.execute()
.unwrap();
```
Or via environment variable:
```bash
UPDATE_SCRIPTS=1 cargo test
```
When enabled, instead of failing on output mismatches, the test files will be updated with actual command output:
**Before (failing test):**
```
exec my-tool --version
stdout "my-tool 1.0"
```
**After running with update mode:**
```
exec my-tool --version
stdout "my-tool 2.1.0"
```
This feature only updates `stdout` and `stderr` expectations while preserving file structure and comments.