imjasonh/testscript-rs
1
0
Fork 0
mirror of https://github.com/imjasonh/testscript-rs synced 2026-07-08 17:16:38 +00:00
Code Activity
23 commits 35 branches 12 tags 334 KiB
Commit graph

7 commits

Author SHA1 Message Date
copilot-swe-agent[bot]
cce8ff1802 Implement support for running specific test files 
Co-authored-by: imjasonh <210737+imjasonh@users.noreply.github.com>
 2025-09-30 07:21:26 +00:00
Copilot
6b5a64703f
Add WorkdirRoot configuration for custom work directories (#27) 
- [x] Add `workdir_root` field to `RunParams` struct
- [x] Add `workdir_root` method to `Builder` API  
- [x] Modify `TestEnvironment::new()` to accept optional root directory
- [x] Update `TestEnvironment::new()` to use
`tempfile::TempDir::new_in()` when root is specified
- [x] Add validation for root directory existence and writability
- [x] Add tests for the new workdir_root functionality
- [x] Update documentation and examples
- [x] Address PR feedback: remove unnecessary files and improve tests
- [x] Fix code formatting with cargo fmt
- [x] Add rustfmt.toml configuration and update CLAUDE.md
- [x] Improve basic test to verify actual workdir usage

## Recent Changes

**Enhanced basic functionality test:**
- Modified `test_workdir_root_basic_functionality` to induce a failure
and use `preserve_work_on_failure(true)`
- Test now verifies that the workdir was actually created in the custom
root directory
- Validates that test files exist in the preserved workdir with expected
content
- **Removed** redundant `test_workdir_root_with_preserve_work` test as
the functionality is now covered by the improved basic test

The test suite is now more meaningful and comprehensive, with 4 focused
tests that provide real validation of the workdir_root functionality:
- Basic functionality with actual workdir verification
- Error handling for nonexistent directories  
- Error handling for non-directory paths
- API chaining validation

All tests pass and the implementation properly validates that custom
workdir roots are used as expected.

<!-- START COPILOT CODING AGENT TIPS -->
---

💡 You can make Copilot smarter by setting up custom instructions,
customizing its development environment and configuring Model Context
Protocol (MCP) servers. Learn more [Copilot coding agent
tips](https://gh.io/copilot-coding-agent-tips) in the docs.

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: imjasonh <210737+imjasonh@users.noreply.github.com>
Co-authored-by: Jason Hall <jason@chainguard.dev>
 2025-09-30 06:57:10 +00:00
Copilot
bfa39f9ab9
Add TestWork functionality to preserve working directories on failure (#24) 
- [x] Analyze current codebase and understand test execution flow
- [x] Add preserve_work_on_failure field to RunParams struct  
- [x] Add preserve_work_on_failure() method to Builder
- [x] Modify TestEnvironment to support preserving work directories
- [x] Update run_script_impl to handle work directory preservation on
failure
- [x] Add comprehensive tests to validate the new functionality  
- [x] Update documentation with usage examples
- [x] Update README with TestWork section showing debugging capabilities
- [x] Fix clippy warnings for code quality
- [x] Validate functionality works correctly in both debug and release
modes
- [x] Fix formatting issues that caused CI failure
- [x] Replace std::mem::forget() with idiomatic TempDir::keep() method

## Improved Code Quality ✅

Replaced the non-idiomatic use of `std::mem::forget()` with the proper
`TempDir::keep()` method for preserving temporary directories. This
approach:

- Is more explicit about the intent to preserve the directory
- Is the recommended way according to tempfile crate documentation  
- Removes the need for manual memory management workarounds
- Makes the code more readable and maintainable

All tests continue to pass and functionality remains unchanged.

<!-- START COPILOT CODING AGENT SUFFIX -->



<details>

<summary>Original prompt</summary>

> 
> ----
> 
> *This section details on the original issue you should resolve*
> 
> <issue_title>Add TestWork functionality to preserve working
directories</issue_title>
> <issue_description>## Feature Request: TestWork Functionality
> 
> Go's testscript supports a `TestWork` parameter that preserves working
directories when tests fail, making debugging much easier.
> 
> ## Proposed API
> 
> ```rust
> testscript::run("testdata")
>     .preserve_work_on_failure(true)
>     .execute()
>     .unwrap();
> ```
> 
> ## Implementation
> 
> - When a test fails and this option is enabled, print the work
directory path
> - Don't clean up the temporary directory on test failure
> - Could also support preserving on success for debugging
> 
> ## Benefits
> 
> - **Easier debugging**: Inspect files created during failed tests
> - **Development workflow**: Understand what went wrong
> - **Compatibility**: Matches Go testscript behavior
> 
> ## Example Output
> 
> ```
> Test failed. Work directory preserved at: /tmp/testscript-work-abc123
> You can inspect the test environment:
>   cd /tmp/testscript-work-abc123
>   ls -la
> ```</issue_description>
> 
> ## Comments on the Issue (you are @copilot in this section)
> 
> <comments>
> </comments>
> 


</details>
Fixes imjasonh/testscript-rs#6

<!-- START COPILOT CODING AGENT TIPS -->
---

💬 Share your feedback on Copilot coding agent for the chance to win a
$200 gift card! Click
[here](https://survey3.medallia.com/?EAHeSx-AP01bZqG0Ld9QLQ) to start
the survey.

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: imjasonh <210737+imjasonh@users.noreply.github.com>
Co-authored-by: Jason Hall <jason@chainguard.dev>
 2025-09-27 19:03:10 +00:00
Copilot
8c3127afea
Add network and advanced condition support with automatic detection and CI optimization (#21) 
## Add Network and Advanced Condition Support with Automatic Detection -
COMPLETE ✅

This PR implements comprehensive network and advanced condition support
for testscript-rs, bringing feature parity with Go's testscript package
with automatic detection like the original.

### Core Features Implemented ✨

#### 1. **Network Conditions (`[net]`) - Always Available &
CI-Optimized**
- Auto-detects network availability with CI-optimized approach:
- **TCP-first detection**: Tries TCP connection to DNS servers (faster
than ping)
  - **Ping fallback**: Uses ping with shorter timeouts if TCP fails
- Cross-platform implementation optimized for CI environments
- Available by default like Go testscript - no manual setup required
- Can be used in test scripts as `[net]` for network-required tests or
`[!net]` to skip when offline

#### 2. **Environment Variable Conditions (`[env:VAR]`)**
- Dynamic condition checking for environment variables at execution time
- No pre-registration required - conditions are evaluated when
encountered
- Supports negation: `[!env:MISSING_VAR]` to test when variables are not
set
- Perfect for CI-specific tests: `[env:CI] exec deploy --dry-run`

#### 3. **Optimized Program Detection - Built-in Like Go**
- **15+ essential programs detected automatically** including:
  - **Basic commands**: cat, echo, ls, mkdir, rm, cp, mv, grep, find
  - **Core development tools**: git, make, curl, python, node, docker  
  - **System tools**: sleep, true, false, sh
- Cross-platform program existence checking (uses `where` on Windows,
`which` on Unix)
- Optimized for CI performance with reduced startup time
- No manual configuration needed - works exactly like Go testscript

### API Simplification

**Simple, Go-like API** - everything detected automatically:
```rust
testscript::run("testdata")
    .execute()  // Network + 15+ programs detected automatically
    .unwrap();
```

**Manual conditions still supported** for custom scenarios:
```rust
testscript::run("testdata")
    .condition("custom", my_custom_check())
    .execute()
    .unwrap();
```

### CI Optimization

- **Network detection**: TCP-first approach with 500ms timeouts (vs
1000ms+ ping)
- **Program list**: Optimized from 35+ to 15+ essential programs for
faster startup
- **Better reliability**: Handles restricted CI network environments
- **Ubuntu CI compatibility**: Resolved timeout and network access
issues

### Real-World Use Cases

This enables powerful testing scenarios that work out of the box:

```
# Network-dependent API tests (automatic detection, CI-optimized)
[net] exec curl https://api.example.com/health
[net] stdout "OK"

# Tool-specific tests (automatic detection)
[exec:docker] exec docker run --rm hello-world
[exec:docker] stdout "Hello from Docker!"

[exec:git] exec git status
[exec:python] exec python --version

# Environment-based behavior
[env:CI] exec deploy --dry-run --verbose
[!env:PRODUCTION] exec run-dangerous-test

# Platform + environment combinations
[unix] [env:DISPLAY] exec gui-test --headless
[windows] [exec:powershell] exec Get-Process
```

### Implementation Details

- **Network detection**: Multi-approach detection (TCP + ping fallback)
optimized for CI reliability
- **Program detection**: Essential program list covering most common
development tools
- **Dynamic evaluation**: Environment conditions are checked at
execution time, not initialization
- **Error handling**: Clear error messages for unknown conditions
- **Cross-platform**: Works consistently across Windows, macOS, and
Linux with CI optimizations

### Go Testscript Compatibility ✅

This implementation now matches Go testscript's behavior:
- All common conditions available by default without setup
- Network condition automatically detected with CI optimization
- Essential program detection built-in with performance focus
- Clean, simple API that "just works"
- Environment variable conditions work dynamically
- CI-friendly performance characteristics

### Testing

Added comprehensive test coverage with 7 new test cases covering:
- Network condition detection (automatic, CI-optimized)
- Environment variable condition parsing and evaluation  
- Optimized program detection (automatic)
- Condition combination scenarios
- Cross-platform compatibility
- Helper function validation

All existing tests continue to pass (47 total), ensuring no regressions.

### Backward Compatibility

This is a purely additive change. All existing functionality is
preserved:
- Platform conditions (`[unix]`, `[windows]`, `[linux]`, `[darwin]`)
- Build type conditions (`[debug]`, `[release]`)
- Manual `.condition()` API still works for custom conditions
- All existing test scripts continue to work unchanged

Fixes imjasonh/testscript-rs#10

<!-- START COPILOT CODING AGENT SUFFIX -->



<details>

<summary>Original prompt</summary>

> 
> ----
> 
> *This section details on the original issue you should resolve*
> 
> <issue_title>Add network and advanced condition support</issue_title>
> <issue_description>## Feature Request: Advanced Condition Support
> 
> Go's testscript supports advanced conditions beyond platform
detection:
> 
> ## Missing Conditions
> 
> 1. **Network conditions**: `[net]` - Test network availability
> 2. **Advanced platform conditions**: More granular OS/arch detection
> 3. **Environment-based conditions**: Custom environment variable
conditions
> 
> ## Proposed API
> 
> ```rust
> testscript::run("testdata")
>     .condition("net", check_network_available())
>     .condition("docker", command_exists("docker"))
>     .condition("env:CI", std::env::var("CI").is_ok())
>     .execute()
>     .unwrap();
> ```
> 
> ## Built-in Condition Helpers
> 
> ```rust
> // Automatic network detection
> testscript::run("testdata")
>     .auto_detect_network()
>     .auto_detect_programs(&["docker", "git", "npm"])
>     .execute()
>     .unwrap();
> ```
> 
> ## Implementation Notes
> 
> - Network detection could ping a reliable host
> - Program detection should use cross-platform approach (not just
`which`)
> - Environment conditions could support pattern matching
> - Consider caching condition results for performance
> 
> ## Use Cases
> 
> - **Docker tests**: `[docker] exec docker run hello-world`
> - **Network tests**: `[net] exec curl example.com`
> - **CI-specific tests**: `[env:CI] exec deploy
--dry-run`</issue_description>
> 
> ## Comments on the Issue (you are @copilot in this section)
> 
> <comments>
> </comments>
> 


</details>
Fixes imjasonh/testscript-rs#10

<!-- START COPILOT CODING AGENT TIPS -->
---

💡 You can make Copilot smarter by setting up custom instructions,
customizing its development environment and configuring Model Context
Protocol (MCP) servers. Learn more [Copilot coding agent
tips](https://gh.io/copilot-coding-agent-tips) in the docs.

---------

Signed-off-by: Jason Hall <jason@chainguard.dev>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: imjasonh <210737+imjasonh@users.noreply.github.com>
Co-authored-by: Jason Hall <jason@chainguard.dev>
 2025-09-27 17:00:36 +00:00
Copilot
fdb53b03fa
Add UpdateScripts functionality for test maintenance (#20) 
- [x] Analyze existing codebase structure and architecture
- [x] Review test files to understand expected UpdateScripts
functionality
- [x] Understand current API and parameter structure
- [x] Add `update_scripts` parameter to RunParams struct
- [x] Add `update_scripts` method to Builder API
- [x] Support UPDATE_SCRIPTS environment variable detection
- [x] Implement update functionality in stdout/stderr command execution
- [x] Modify output comparison to capture and update actual output when
in update mode
- [x] Add script file modification logic to update test files
- [x] Create focused tests for update functionality
- [x] Validate changes work end-to-end
- [x] Update documentation in README.md
- [x] Clean up temporary files
- [x] Move UpdateScripts section to bottom of README per review feedback
- [x] Fix code formatting issues with cargo fmt

## Implementation Summary

Successfully implemented complete UpdateScripts functionality for test
maintenance in testscript-rs:

### Key Features
- **API support**: `.update_scripts(true)` method on Builder
- **Environment variable**: `UPDATE_SCRIPTS=1` environment variable
support
- **Smart updating**: Only updates `stdout`/`stderr` expectations,
preserves file structure
- **Proper quoting**: Handles complex output with appropriate shell
quoting
- **Error handling**: Continues execution instead of failing in update
mode

### API Usage
```rust
// Via API
testscript::run("testdata")
    .update_scripts(true)
    .execute()
    .unwrap();

// Via environment variable  
UPDATE_SCRIPTS=1 cargo test
```

### Implementation Details
- Added `update_scripts: bool` field to `RunParams`
- Modified `run_script_impl` to collect updates instead of failing on
mismatches
- Implemented `apply_script_updates` function for file modification
- Added comprehensive test coverage with 4 test cases
- Updated README.md with usage examples (moved to bottom per review
feedback)
- Fixed code formatting issues identified by CI

### Testing
- All existing tests continue to pass
- New functionality tested via API and environment variable
- Manual validation confirms proper file updating
- Normal mode behavior preserved (still fails on mismatches when
disabled)
- Code passes formatting checks and linting

The implementation matches Go's testscript behavior and provides
essential test maintenance functionality.

<!-- START COPILOT CODING AGENT SUFFIX -->



<details>

<summary>Original prompt</summary>

> 
> ----
> 
> *This section details on the original issue you should resolve*
> 
> <issue_title>Add UpdateScripts functionality for test
maintenance</issue_title>
> <issue_description>## Feature Request: UpdateScripts (Test Maintenance
Mode)
> 
> Go's testscript supports an `UpdateScripts` parameter that
automatically updates test files with actual command output. This is
invaluable for test maintenance.
> 
> ## Proposed API
> 
> ```rust
> testscript::run("testdata")
>     .update_scripts(true)
>     .execute()
>     .unwrap();
> ```
> 
> Or via environment variable:
> ```bash
> UPDATE_SCRIPTS=1 cargo test
> ```
> 
> ## How It Works
> 
> When enabled, instead of failing on output mismatches:
> 1. **Capture actual output**
> 2. **Update the test file** with the actual output
> 3. **Continue to next test**
> 
> ## Example
> 
> Before (failing test):
> ```
> exec my-tool --version
> stdout "my-tool 1.0"
> ```
> 
> After running with update mode:
> ```
> exec my-tool --version
> stdout "my-tool 2.1.0"
> ```
> 
> ## Benefits
> 
> - **Easy test maintenance**: Update expected outputs after changes
> - **Reduce manual work**: No hand-editing of test files
> - **Go compatibility**: Matches upstream behavior
> - **Development workflow**: Essential for evolving CLI tools
> 
> ## Implementation Notes
> 
> - Should only update `stdout`/`stderr` expectations
> - Preserve comments and file structure
> - Add safety checks to prevent accidental
overwrites</issue_description>
> 
> ## Comments on the Issue (you are @copilot in this section)
> 
> <comments>
> </comments>
> 


</details>
Fixes imjasonh/testscript-rs#8

<!-- START COPILOT CODING AGENT TIPS -->
---

💬 Share your feedback on Copilot coding agent for the chance to win a
$200 gift card! Click
[here](https://survey3.medallia.com/?EAHeSx-AP01bZqG0Ld9QLQ) to start
the survey.

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: imjasonh <210737+imjasonh@users.noreply.github.com>
 2025-09-26 21:47:09 -04:00
Jason Hall
0c4315ece8
Refactor run module into modular structure (#18) 
## Summary

Refactors the monolithic `run.rs` file into a clean modular structure to
improve maintainability and code organization.

### Changes

- **Split `run.rs` into focused modules:**
  - `run/environment.rs` - TestEnvironment and file operations
  - `run/params.rs` - RunParams configuration and conditions  
  - `run/commands.rs` - Built-in command implementations
  - `run/execution.rs` - Command dispatch and execution logic
  - `run/mod.rs` - Public API coordination

### Benefits

- **Better maintainability** - Each file has clear responsibility
- **Easier navigation** - Find specific functionality quickly
- **Separation of concerns** - Commands, config, environment isolated
- **Prepares for future features** - Clean foundation for WorkdirRoot
and other enhancements

### Test Plan

- [x] All 37 tests still pass
- [x] No functionality changes
- [x] Clean compilation with no warnings
- [x] Pre-commit hooks pass (formatting, linting, compilation)

This is a pure refactoring with no behavior changes - just improved code
organization.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-authored-by: Claude <noreply@anthropic.com>
 2025-09-27 01:17:45 +00:00
Jason Hall
c39114f1a9 initial commit 
Signed-off-by: Jason Hall <jason@chainguard.dev>
 2025-09-26 18:41:14 -04:00