<div align="center">

🧪 dsh-test-drive

• 1024 store channel: npm i -g dsh1024 once, then dsh1024 plugin --profile web add dsh-test-drive (counts toward the deepseek1024.com install ranking).

Isolated install-and-smoke test drives for DeepSeek Harness plugins.

Install, smoke, verify, and clean up in a throwaway profile — your real ~/.dsh stays untouched.

English · 简体中文 · Español · Português · हिन्दी

</div>

---

Compatibility

Component	Version
DeepSeek Harness	dsh-v0.1.3-alpha.1 (GitHub tag, verified 2026-09-06: full gate chain + profile install smoke). npm dependency line 0.1.2-rc.1 (peer dependencies >=0.1.2-rc.1 <0.2.0)
Node.js	^22.19.0 || >=24.0.0
Package manager	pnpm@11.7.0
Platform	Windows / macOS / Linux (host-only plugin)
External tools	dsh CLI on PATH (auto-detected, npm shims parsed), pnpm on PATH

What you get

• test_drive tool — one target through the complete pipeline: dsh plugin add → --dump-config patch check → headless boot smoke (FAILED-marker scan + optional one-shot task) → optional capability assertion → dsh plugin remove → quarantined cleanup. Returns the structured record synchronously, or { kind: 'background', jobId } with background: true.
• /testdrive command — batch drive of a whitespace/comma-separated target list as a drive-batch background job over ctx.jobs, producing a matrix report (JSON + Markdown).
• drive_report tool — fetch any stored run (tdr_...), matrix (tdm_...), or the latest matrix; rendered as Markdown.
• Capability assertion — beyond "booted and exited": the optional capability stage drives one headless task that calls a named tool (or runs a /command) and verifies the durable session log recorded the invocation and the observed output contains expect. A clean boot is only a smoke test; observed proves a named capability really works.
• Structured results — every record carries the discriminator schema: "dsh-test-drive/v1" with first-class fields: stages.install.status (pass/fail), stages.smoke.status (pass/fail/boot-ok/skipped), stages.capability.status (observed/invoked/not-registered/skipped/failed), per-stage durationMs, sanitized summary/outputTail, and an overall verdict (pass/fail/partial/unknown). This is the machine-readable contract downstream scorers (dsh-score) consume.
• Safety by construction — every temp directory is created by this plugin under a dedicated dsh-test-drive- prefix, tracked in a live ownership registry, and removed only through a dry-run → quarantine-rename → delete ladder. The host profile is never read or written.

Quick start

Git channel

dsh plugin --profile web add github:PerryLink/dsh-test-drive#<commit-sha>

The first add fails because pnpm blocks the package's prepare build; copy the exact key pnpm printed into the profile's pnpm-workspace.yaml and re-run:

allowBuilds:
  'dsh-test-drive': true

npm channel

dsh plugin --profile web add dsh-test-drive

Prebuilt packages need no build allowance. Restart the profile, then use test_drive / /testdrive from a session.

Install & uninstall

dsh plugin --profile web add dsh-test-drive     # install (npm) — or the git form above
dsh plugin --profile web remove dsh-test-drive  # uninstall

Configuration

All keys are optional (defaults shown); invalid values fail loudly at load.

Key	Default	Description
profileName	headless	Profile template initialized inside each throwaway DSH_HOME (base + headless bundles).
dshBin	""	Absolute dsh executable override; empty auto-detects dsh on PATH.
headlessTask	"Reply with exactly: ok"	One-shot task for the boot-smoke stage; empty skips the stage.
forwardEnv	[]	Environment VARIABLE NAMES (never values) forwarded into test-profile child processes.
allowBuilds	true	Allowlist a blocked git prepare build in the test profile and retry the install once.
installTimeoutMs	600000	dsh plugin add stage deadline.
configTimeoutMs	60000	--dump-config stage deadline.
smokeTimeoutMs	300000	Headless boot-smoke stage deadline.
capabilityTimeoutMs	300000	Capability-assertion task deadline.
capability.enabled	false	Run the capability-assertion stage (registered → invoked → observed).
capability.kind	tool	What to assert: tool or command.
capability.name	""	Tool or command name (no leading /).
capability.args	""	Invocation text: tool arguments (JSON-ish) or command words.
capability.expect	""	Literal expected in the observed output (case-insensitive substring).
uninstallTimeoutMs	120000	dsh plugin remove stage deadline.
outputTailBytes	8000	Cap on the sanitized output tail recorded per stage.
keepTempDirs	false	Keep temp dirs on failure for forensics (ownership is dropped; you clean up).
maxBatchTargets	20	/testdrive batch cap.
batchConcurrency	1	Batch concurrency (serial avoids pnpm-store contention).

Tools & surfaces

test_drive

test_drive(target: string, headlessTask?: string, background?: boolean,
           capability?: { kind: 'tool' | 'command', name: string,
                          args: string, expect: string })

• target — git spec (github:owner/repo#sha, git+https://...), npm name, local path, or .tgz tarball.
• capability — assertion after the boot smoke: the agent calls name (tool) or runs /name (command) with args; the stage reads the durable session log and requires the observed output to contain expect. Needs DEEPSEEK_API_KEY (host env or forwardEnv); without it the stage is skipped, never failed.
• Returns the full structured record; see the sample below.
• background: true starts a drive-batch job and returns its id.

/testdrive <targets...>

Starts one background batch job; progress streams through the job output, and the final line names the matrix id for drive_report.

drive_report(id?)

Returns a run record (tdr_...), a matrix (tdm_...), or — with no id — the latest matrix.

Structured result sample

{
  "schema": "dsh-test-drive/v1",
  "run": { "runId": "tdr_9f2c...", "startedAt": "2026-08-16T00:00:00.000Z",
           "finishedAt": "2026-08-16T00:00:45.120Z", "durationMs": 45120,
           "harnessVersion": "0.1.2-rc.1", "pluginVersion": "0.3.6",
           "platform": "win32", "node": "v22.22.3" },
  "target": { "kind": "repo", "spec": "github:owner/dsh-click#abc123",
              "resolved": { "packageName": "dsh-click", "packageVersion": "0.1.0",
                            "hasBundleManifest": true } },
  "isolation": { "tempDshHome": true, "tempWorkspace": true, "tempStore": true,
                 "hostHomeTouched": false },
  "stages": {
    "install":   { "status": "pass", "exitCode": 0, "durationMs": 30412, "attempts": 2,
                   "summary": "install ok after allowBuilds allowance", "outputTail": "",
                   "allowBuildsNeeded": true },
    "config":    { "status": "pass", "exitCode": 0, "durationMs": 2310, "attempts": 1,
                   "summary": "dump ok (exit 0)", "outputTail": "",
                   "patchEffective": true, "layers": ["dsh-click"] },
    "smoke":     { "status": "boot-ok", "exitCode": 1, "durationMs": 4123, "attempts": 1,
                   "summary": "booted without loader failures; headless task did not complete (credentials/model unreachable)",
                   "outputTail": "", "bootFailed": false, "taskCompleted": false },
    "capability": { "status": "observed", "exitCode": 0, "durationMs": 8123, "attempts": 1,
                    "summary": "tool \"plugin_vet\" called and its result contains the expectation",
                    "outputTail": "", "capabilityKind": "tool", "name": "plugin_vet",
                    "expectMatched": true,
                    "detail": "tool \"plugin_vet\" called and its result contains the expectation" },
    "uninstall": { "status": "pass", "exitCode": 0, "durationMs": 5123, "attempts": 1,
                   "summary": "remove ok (exit 0)", "outputTail": "" },
    "cleanup":   { "status": "pass", "quarantined": true, "removed": true,
                   "summary": "owned temp root quarantined and removed" }
  },
  "verdict": "pass",
  "verdictReason": "install, patch, boot, and uninstall verified; headless task inconclusive (see smoke.summary)"
}

Verdict rules: install failure, boot failure (smoke.fail), or a capability stage that reached not-registered/failed ⇒ fail; install pass + patch effective + clean boot (pass/boot-ok) + uninstall pass ⇒ pass (with a capability note when observed); anything installed but missing a later assurance ⇒ partial; otherwise unknown.

CI (GitHub Actions)

The repository ships a composite action.yml that reuses dsh-test-drive in any plugin repo. It drives a target in an isolated throwaway profile and emits the report pair CI consumes: Markdown (PR comment) and JUnit XML (test reporter / status check).

# .github/workflows/test-drive.yml
name: test-drive
on: [pull_request, workflow_dispatch]
jobs:
  smoke:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Drive this plugin
        id: drive
        uses: PerryLink/dsh-test-drive@v0.2.4
        with:
          target: github:${{ github.repository }}#${{ github.sha }}
      - name: Publish JUnit
        uses: EnricoMi/publish-unit-test-result-action@v2
        with:
          files: ${{ steps.drive.outputs.junit }}
      - name: Comment the report
        run: cat "${{ steps.drive.outputs.markdown }}"

action.yml inputs: target (required), headless-task (optional smoke task), dsh-version (the dsh CLI spec). Outputs: markdown, junit (report paths), and verdict (pass/fail/partial/unknown). The drive itself stays keyless (install → patch check → boot smoke → uninstall → cleanup); a capability assertion is the only stage that needs DEEPSEEK_API_KEY, and it is skipped, never failed, without one.

The same report pair is available programmatically: renderDriveResult / renderMatrix (Markdown) and renderDriveJUnitXml / renderMatrixJUnitXml (JUnit) are exported from the package root over a settled DriveResult or MatrixRecord.

Permissions & data

• Only public services are consumed: ctx.subprocess, ctx.jobs, ctx.storageDomain, ctx.tools, ctx.commands.
• Reports are stored in the test_drive storage-domain (tables runs, matrices; latest-matrix pointer). When the composition has no storageDomain (e.g. the shipped headless profile), tools still work and report persistence is disabled with a logged reason.
• Child processes inherit a credential-scrubbed environment: host secrets never reach a tested profile unless you explicitly name them in forwardEnv. Values are never logged.
• All report/log strings pass through pure sanitizers: token literals, URL credentials, and bearer headers are redacted, temp-root paths are replaced with <testdrive-temp>, and tails are byte-capped.

Security boundaries

• Isolation. Each drive runs inside a fresh mkdtemp root under the OS temp dir: a throwaway DSH_HOME, a throwaway working directory, and a redirected pnpm store. The tested plugin's code only ever runs in that profile; your host profile is untouched.
• Ownership. A live registry records every root this plugin instance creates. Cleanup refuses anything that is not a registered direct child of the OS temp dir carrying the dsh-test-drive- prefix — no %TEMP% sweeps, no foreign prefixes, no real-home paths.
• Cleanup ladder. Before any mutation the full dry-run plan is logged (absolute paths). Removal renames the root into a dsh-test-drive-quarantine-<ts> directory first, verifies, then deletes; failures leave the directory quarantined and reported, never silently dropped. Cleanup runs in a finally on success, failure, timeout, and abort, and again on plugin teardown.
• allowBuilds is a real permission. Allowing a git package's prepare build executes that package's code at install time. The allowance is scoped to the throwaway profile only, but only test targets you trust, and pin commits.
• Headless smoke is keyless by default. The boot check needs no credentials; completing the one-shot task does. Forward credentials explicitly (forwardEnv) and never log them.

Known limitations

• Installing registry/git targets requires network access from the child dsh/pnpm processes.
• The smoke task needs model credentials to reach pass; without them it reports the honest boot-ok.
• In compositions without storageDomain, reports are not persisted (drive_report fails honestly).
• dsh must be locatable on PATH (or set dshBin); on Windows the npm .cmd/.bat shim is parsed automatically, a bare .ps1 resolution asks for dshBin.
• Batches default to serial execution; raising batchConcurrency shares the pnpm-store disk, not correctness.

Development

pnpm install
pnpm run typecheck && pnpm run typecheck:ci && pnpm test
pnpm run build && pnpm run verify:self-contained && pnpm run verify:artifacts && pnpm pack

• typecheck resolves @deepseek-ai/* through the local harness checkout; typecheck:ci checks against the published 0.1.2-rc.1 types.
• Tests use the real Context/Session/ToolRuntime/LocalJobRegistry/storage stack with a scripted subprocess provider.
• Real-CLI end-to-end (requires network + dsh on PATH): DSH_TESTDRIVE_E2E=1 pnpm run test:e2e — drives this package's own checkout through the real install-smoke loop.
• Release: node scripts/release.mjs <x.y.z> (bumps, stamps CHANGELOG, re-runs the gate, commits + tags; never pushes).

Topics

dsh, dsh-plugin, deepseek-harness, deepseek, cordis, plugin-testing, install-smoke, compatibility-matrix, ci

Contributors

PerryLink — design and implementation.

PerryLink DSH Plugin Family

This project is one of the 33 DeepSeek Harness plugins maintained by PerryLink. If this one helps you, the others likely will too:

Plugin	One-liner
dsh-dsh-auto-review	Second-model auto-review on the approval chain, fail-closed by default
dsh-dsh-background-agents	Durable background child agents with a Web UI sidebar, messaging and interrupt
dsh-dsh-budget	Cost governance for DeepSeek Harness: budgets, carbon, and latency in one panel.
dsh-dsh-checkpoint-rewind	Claude Code /rewind-equivalent: snapshots, session forks, one-shot restore
dsh-dsh-claude-move	Migrate Claude Code sessions, memory, skills and CLAUDE.md into DSH
dsh-dsh-click	Cross-platform native desktop control for DeepSeek Harness — Windows first.
dsh-dsh-composer-history	Terminal-style input history for the web composer: arrows, Ctrl+R search
dsh-dsh-data-quality	Dataset quality checks and citation cross-checks (the optional numeric bridge consumed here)
dsh-dsh-defend	Prompt-injection, jailbreak, and secret-leak defense for DeepSeek Harness.
dsh-dsh-doublecheck	Engineering-discipline guard: requirements grill, test gates, adversary review
dsh-dsh-draw	Unified static-image generation routing for DeepSeek Harness.
dsh-dsh-fast	Read-only performance diagnostics for DeepSeek Harness.
dsh-dsh-fund-research	Deterministic research reports for Chinese public mutual funds
dsh-dsh-github	GitHub PR/issues integration for DSH, every write gated by approval
dsh-dsh-industry-research	Industry research orchestration that seals its deliverables through this plugin's ctx.researchReport.assemble
dsh-dsh-library	Local document knowledge base for DeepSeek Harness.
dsh-dsh-local-ai	Local-model (Ollama) integration for DeepSeek Harness.
dsh-dsh-lsp-actions	LSP diagnostics, formatting, completion, code actions and rename over language servers
dsh-dsh-mask	PII masking middleware: anonymize at the model boundary, restore at the display layer
dsh-dsh-mcp-panel	Read-only MCP runtime panel: /mcp command + Settings tab with status, tools and errors
dsh-dsh-memento	Approval-gated cross-session memory: ctx.memory seam + SQLite + memory tool
dsh-dsh-observe	OpenTelemetry and Langfuse observability exporter for DeepSeek Harness.
dsh-dsh-output-styles	Claude Code outputStyles-equivalent runtime style switching
dsh-dsh-permission-rules	Claude Code-style declarative allow/deny/ask permission rules with audit
dsh-dsh-plugin-guide	Plugin-development knowledge base as an on-demand agent skill
dsh-dsh-research-report	Verifiable research-report engine: content-addressed evidence ledger and sealed versions
dsh-dsh-score	Multi-dimensional quality scoring for DeepSeek Harness plugins.
dsh-dsh-session-pin	Pin sessions in the Web sidebar with durable ordering
dsh-dsh-session-sync	Cross-device session sync for DeepSeek Harness — a dedicated git mirror of your session store.
dsh-dsh-skill-pack-security	Security-audit skill pack: secret scan, dependency and supply-chain review
dsh-dsh-talk	Voice-first session loop for DeepSeek Harness: talk to it, hear it answer.
dsh-dsh-translate	Vendor parameter translation and deterministic JSON repair for DeepSeek Harness.

Install from the DSH Desktop Market

All PerryLink plugins are browsable in the built-in DSH Desktop Market: Market → Sources → add source → paste https://perrylink-dsh-catalog.perrylink.workers.dev/catalog-source.json → select it. Installation still goes through the Market's npm-identity verification and your confirmation.

License

Apache-2.0