Skip to content

host-testing.md

Latest commit

 

History

History
117 lines (103 loc) · 7.03 KB

host-testing.md

File metadata and controls

117 lines (103 loc) · 7.03 KB
Raw

MTP Host Testing (Agent Digest)

RevitDevTool.TestAdapter (NuGet) is the only supported test integration. NUnit is the default engine, TUnit is opt-in. Product: docs/product/host-testing.md · docs/product/tunit-host-testing.md. Structure: docs/architecture/Testing/README.md. Run/author tests: .agents/skills/revit-test/SKILL.md.

Main flow

test csproj (HostName, HostVersion, NUnit|TUnit)
  -> package targets: Reference the selected DevTools.{NUnit|TUnit}.MTP.dll
     + Abstractions, write discovery-refs.txt and [AssemblyName].testconfig.json
  -> generated MTP entry point calls adapter hook + selected sibling hook
  -> discovery: local ExploreTests / TUnit catalog in the testhost, no host process
  -> run: TestRunner.exe locates or launches the host, then testing/hello|run|cancel

Adapter writes devtools.frameworkId to testconfig.json. Missing frameworkId on a hand-written devtools section is a build Error. Sibling selection is MSBuild (TestingFramework → one Reference + builder hook), not runtime plugin keys.

Verify

# 1. compile + in-repo tests (adapter unit tests still ProjectReference)
dotnet build source/DevTools.Testing.Host/DevTools.Testing.Host.csproj -c Debug
dotnet run --project tests/DevTools.TestAdapter.Tests/DevTools.TestAdapter.Tests.csproj
dotnet run --project tests/DevTools.NUnit.MTP.Tests/DevTools.NUnit.MTP.Tests.csproj
# TestRunner.Tests is MSTest.Sdk 4.4 (MTP testhost), not xunit
dotnet run --project tests/DevTools.TestRunner.Tests/DevTools.TestRunner.Tests.csproj

# 2. pack to output/nuget + drop stale global-packages extraction (default)
scripts/pack-test-adapter.ps1

# 3. package surface (restore from that nupkg + host-free discovery), net48 / net8 / net10
scripts/test-adapter-matrix.ps1

# 4. live (host running; root global.json is MTP)
dotnet test --project samples/DevTools.NUnit.SampleTests/DevTools.NUnit.SampleTests.csproj -c Debug.Autodesk.2026 --filter Arithmetic_runs_inside_host

Host DLL changes: scripts/build-host.ps1 -Year <year>. Runner: dotnet publish source/DevTools.TestRunner -c Release. Adapter nupkg: scripts/pack-test-adapter.ps1 (not scripts/pack.ps1). The pack script deletes %USERPROFILE%\.nuget\packages\revitdevtool.testadapter\<version> so the next sample restore cannot keep a previous extraction of the packed version.

Rules

  • Consumer properties: HostName, HostVersion, ForceLaunch, PerTestTimeout, LaunchTimeout, TestingFramework (nunit default, tunit opt-in). UseRevit / UseAutoCad are this repo's sample compile flags, not package settings.
  • Do not set RuntimeIdentifier on net8 / net10 test projects. The package sets AppendRuntimeIdentifierToOutputPath=false. A RID on net8 (Revit 2025) nests testhost output under win-x64. Samples set win-x64 only when TargetFramework starts with net4.
  • net48 + TUnit needs no Polyfill. The package defaults EnableTUnitPolyfills=false and compiles build/netfx/ModuleInitializerAttribute.cs into the project, skipping it when a Polyfill reference or another ModuleInitializerAttribute.cs is already there. Opt out with NetFxModuleInitializer=false only when that skip misses the existing type (see docs/product/tunit-host-testing.md).
  • The packaged build/*.props|targets must not read $(Configuration), repo paths, or ProjectReference. In-repo samples restore RevitDevTool.TestAdapter from output/nuget (same nupkg an end user gets). There is no checkout-only Local.targets path.
  • --filter is the method-name option (NUnit <name re="1"> regex). --filter-uid is the TestNode uid (ITest.FullName; TestName/SetName is Class.Method("DisplayName")). --list-tests text prints DisplayName — never paste a text line as --filter-uid. TestRunner does not discover tests.
  • MTP samples: dotnet test --project samples/DevTools.*.SampleTests/… from the repo root (root global.json is MTP). samples/ricaun.NUnit.SampleTests overrides to VSTest — cd into that folder. In-repo tests/: dotnet run --project tests/<proj>/<proj>.csproj still works; dotnet test --project tests/<proj>/<proj>.csproj from root is now MTP too. Never VSTest --filter FullyQualifiedName~ on MTP projects.
  • Use an Autodesk configuration (Debug.Autodesk.2025, …). Plain Debug does not set RevitVersion / TargetFramework, so the sample does not build and Test Explorer falls back to a source tree that is not MTP discovery.
  • Autodesk configs flatten host obj/bin; MTP keeps AppendTargetFrameworkToOutputPath=true so its three TFMs never share a folder (CS2012 / MSB3713). Do not collapse TargetFrameworks on packable projects.

Traps

  • Discovery must stay host-free: Test Explorer refresh and --list-tests must not start a host, and must not read testconfig.json host options. ForceLaunch=false still starts a matching-version host on run when none is open.
  • "Test discovery aborted: 0 Tests found" = the testhost died in static init or Discover. Usual cause is a timestamp-stale DevTools.*.MTP.dll beside the test exe (TypeLoadException on an ITestDiscoverer member). Rebuild the test project, not only the host year; the sibling copy runs with SkipUnchangedFiles=false.
  • Do not use NUnit.Engine in the host, and never add NUnit3TestAdapter to a host-test project. samples/ricaun.NUnit.SampleTests is the third-party VSTest comparison sample over the same HostSmokeTests; its explorer tree is not a DevTools discovery bug. Intentional_failure_for_demo is an expected Assert.Fail.
  • net48 "could not be discovered" = testhost failed to bind Unsafe 6.0. That BCL comes from the adapter's Microsoft.Testing.Platform.MSBuild graph plus AutoGenerateBindingRedirects; the nupkg ships no loose 3rd-party DLLs.
  • net48 has no load context: if the host already loaded an assembly with the same identity, in-host tests bind that copy, not the generation snapshot. Restart the host after deploying, or use net8+.
  • Stream-loaded assemblies have an empty Assembly.Location. Tests that locate assets use NUnit TestContext.WorkDirectory (the generation shadow).
  • Live testing/run is marshaled through IHostContextExecutor (NUnit RunOnMainThread). WPF Dispatcher.Invoke is not a Revit API context.
  • Do not add a Host TraceListener or ILogger dump of CaseResult.Output; Trace/Debug already fan out and Console is write-through at case finish (0017).
  • TUnit in-host MissingMethodException on ClientInfoService: MTP 2.4.0 needs IClientInfo + IClientCapabilities; the Runtime registers TUnitEngineClientInfo and ships in the installer, so redeploy the host (not the nupkg).
  • Visual Studio Debug attaches the testhost, then the Runner EnvDTE-attaches that VS instance to the host (0025). Stop Debugging during launch kills only a host this run spawned. Rider / C# Dev Kit: attach the host PID, then Run. Runner host year is --host-version.