[skipci]Merge pull request #2977 from milutinke/skills/integration-testing-improvements

.skills/mcc-integration-testing/SKILL.md

@@ -1,11 +1,50 @@
 ---
 name: mcc-integration-testing
-description: Repeatable real-server integration testing for Minecraft Console Client against a local offline Minecraft Java server. Use this whenever the user wants to confirm nothing broke, validate runtime or protocol changes end-to-end, exercise movement, physics, inventory, entity handling, or run a single-version or cross-version MCC regression sweep on a real server.
+description: >-
+  Use when proving MCC behavior on a real local Minecraft server, validating
+  runtime or protocol changes end-to-end, exercising movement, physics,
+  inventory, entity, chat, or terrain behavior, or running a single-version or
+  cross-version regression sweep.
+metadata:
+  category: discipline
+  triggers:
+    - integration test
+    - real server
+    - local server
+    - regression sweep
+    - rcon
+    - tmux
+    - offline mode
+    - online mode
+    - movement
+    - physics
+    - inventory
+    - entity
+    - terrain
+    - chat
 ---
 
 # MCC Integration Testing
 
-Use this skill when the task is "prove it still works on a real server", not just "reason about whether it should work."
+Use this skill when the task is "prove it on a real server", not just "reason about whether it should work."
+
+Read [references/online-mode.md](references/online-mode.md) when the user asks for Microsoft login, device-code auth, or an online-mode server run. Use [references/command-matrix.md](references/command-matrix.md) for stable MCC-side and RCON-side commands.
+
+## Iron Law
+
+Only say MCC was integration tested when MCC ran against a real local server and the claim is backed by real MCC output plus real server logs.
+
+Calling build-only, reasoning-only, or join-only work "integration tested" is a rules violation, not shorthand.
+
+These do not count as end-to-end proof:
+
+- static reasoning, source comparison, or build success
+- join or login success by itself
+- a long-lived idle connection by itself
+- a grep that only says there were no errors
+- testing one shared-route version and silently claiming adjacent versions also passed
+
+If the environment cannot run a real server, say so and report the result as unexecuted or inferred, not integration tested.
 
 ## Default target
 
@@ -15,10 +54,19 @@ Use this skill when the task is "prove it still works on a real server", not jus
 ## Guardrails
 
 - Use a real local server.
+- Launch MCC against an explicit `localhost:<server-port>` target for repeatable local tests.
 - Keep version matrices sequential in shared local environments. The tmux server harness is shared state by default.
 - Prefer temporary MCC configs for scripted runs so one test does not contaminate the next.
+- Default to offline auth in generated temp configs. Do not trust the repo-root `MinecraftClient.ini` account defaults.
+- If the user explicitly asks for Microsoft online login, honor that request and generate the temp config for Microsoft auth instead of offline mode.
+- For Microsoft auth, prefer an interactive TTY launch with `BasicIO-NoColor` so the device code is easy to read and relay to the user.
+- Do not use file-input mode during Microsoft auth. Launch interactively first, complete login, then switch to scripted control only if needed.
+- For online-mode tests, prefer a clean temp config with no join-time bots or scheduled tasks. Inherited `ScriptScheduler` or `DiscordRpc` settings can pollute the session and send unintended chat right after login.
 - Legacy and modern command syntax differ. Do not assume one server-command profile fits every version.
 - Use actual MCC output and actual server logs for assertions. Do not invent success strings.
+- Treat server `Done` as startup progress, not RCON readiness. Retry the first RCON command before assuming the setup is broken.
+- If a change touches shared routing or a version range, test at least one adjacent version that shares that path, or explicitly mark adjacent versions as unexecuted and inferred.
+- For palette or version-content changes, probe at least one neighboring or existing item, entity, or block. Do not only check the headline addition.
 
 ## Choose the test mode
 
@@ -46,6 +94,8 @@ Use this when the user asks for a regression sweep in a strict scenario order su
 - effects
 - inventory
 
+Broad validation should usually cover `connect-test`, `item-test`, `entity-test`, `terrain-test`, and `chat-test`.
+
 Command:
 
 ```bash
@@ -78,10 +128,23 @@ Offline configuration helper:
 .skills/mcc-integration-testing/scripts/ensure_offline_server.sh 1.21.11-Vanilla
 ```
 
+By default, the config helper prepares offline auth. To opt into another auth mode for a specific run, set:
+
+```bash
+MCC_TEST_ACCOUNT_TYPE=microsoft
+MCC_TEST_PASSWORD=
+```
+
+Optionally override the login name with the fourth argument to the config helper.
+
 ## Scripts and tools
 
 - `.skills/mcc-integration-testing/scripts/ensure_offline_server.sh`
   - configures persistent offline mode and RCON
+- `.skills/mcc-integration-testing/scripts/prepare_offline_mcc_config.sh`
+  - copies `MinecraftClient.ini`, prepares offline login by default, and can switch to Microsoft auth when explicitly requested
+- `.skills/mcc-integration-testing/scripts/get_server_port.sh`
+  - resolves the actual local server port from `server.properties` or the latest server log
 - `.skills/mcc-integration-testing/scripts/run_full_spectrum_test.sh`
   - single-version deep smoke with built-in assertions
 - `.skills/mcc-integration-testing/scripts/summarize_test_run.sh`
@@ -89,20 +152,57 @@ Offline configuration helper:
 - `tools/run-creative-e2e.sh`
   - ordered creative-mode E2E regression scenario
 
+## Evidence Discipline
+
+In every report, separate:
+
+- `Executed`: exact scripts, commands, versions, auth mode, and whether the run was sequential or single-version
+- `Observed`: exact MCC output, exact server-log evidence, and the saved log directory
+- `Inferred`: conclusions not directly shown by that run's runtime evidence
+
+Never upgrade inferred claims to observed facts. Absence of errors is supporting evidence only; pair it with a positive assertion for the feature under test.
+
+## Red Flags
+
+Stop and fix the test plan if you are about to:
+
+- claim movement, inventory, entity, terrain, physics, or chat coverage from join success alone
+- reuse repo-root `MinecraftClient.ini` or another user-local stateful config
+- run multi-version tests in parallel in a shared tmux or shared server environment
+- let inherited bots, schedulers, or other user-local noise send chat or commands during validation
+
 ## What to report back
 
 Always summarize:
 
 - which version or versions were tested
-- which scenario was used
+- which port or ports were used
+- which auth mode and scenario were used
 - whether the run was sequential or single-version
+- the exact scripts or commands executed
 - pass or fail per major phase
 - concrete evidence from MCC and server logs
 - the saved log directory
+- what was not executed and what remains inferred
+- which adjacent versions were not run but were mentioned
+
+## When Not to Use
+
+- build-only verification
+- static protocol or source comparison with no real server run
+- documentation or prompt work
+- code review requests that do not ask for executed runtime proof
 
 ## Troubleshooting
 
 - If the first RCON command fails, retry it before assuming the setup is broken.
+- If MCC reaches Microsoft device-code login during an offline test, stop and inspect the generated temp config before retrying.
+- If the user explicitly requests Microsoft online login, set `MCC_TEST_ACCOUNT_TYPE=microsoft` before launching the harness.
+- If the user explicitly requests Microsoft online login, use `BasicIO-NoColor` in a real TTY, relay the device code from the TUI, and avoid pressing empty Enter at any auth prompt.
+- If the online-mode session sends unexpected chat or commands right after join, inspect inherited bot settings first. `ChatBot.ScriptScheduler` tasks and `ChatBot.DiscordRpc` are common sources of test noise in user-local configs.
+- If `dotnet run` cannot see an existing Microsoft session, check whether `SessionCache.db` and `ProfileKeyCache.ini` need to be synced from `MinecraftClient/bin/Release/net10.0/` to the repo root.
+- If Microsoft auth keeps prompting even with a valid session cache, verify `Account.Login` matches the cached username exactly.
+- If MCC reports `Connection refused`, verify the launched target matches the server's actual `server-port`.
 - If multiple versions are being tested, do not start them in parallel unless the harness isolates tmux sessions and input files.
 - If a test assertion fails, inspect the real MCC output before changing the code or weakening the assertion.
 - If an older server behaves oddly on Linux, check `use-native-transport=false` in `server.properties`.

.skills/mcc-integration-testing/references/online-mode.md

@@ -0,0 +1,48 @@
+# Online-Mode Notes
+
+Use this flow only when the user explicitly asks for Microsoft login or wants to validate against an online-mode server.
+
+## Launch mode
+
+- Prefer `BasicIO-NoColor` in a real TTY so the Microsoft device-code prompt is easy to read and copy.
+- Do not use `MCC_FILE_INPUT=1` during the auth step. It is for scripted command injection, not interactive login.
+- Avoid `nohup`. Use `tmux` for long-running sessions that still need a TTY.
+- Start from a clean temp config when possible. If the temp config is copied from a user-local `MinecraftClient.ini`, inspect `ChatBot.ScriptScheduler` and `ChatBot.DiscordRpc` before the run.
+
+## Session cache behavior
+
+- `dotnet run --project MinecraftClient ...` uses the repo root for `SessionCache.db` and `ProfileKeyCache.ini`.
+- The compiled binary under `MinecraftClient/bin/<Config>/net10.0/` uses that output directory instead.
+- If a session exists in one location and not the other, sync the cache files before assuming login is broken.
+
+## Account settings
+
+- `Account.Login` must be populated for MCC to look up a cached Microsoft session.
+- The cached key is the username form MCC stored, typically the lowercase username, not necessarily the email address.
+- MCC rewrites `MinecraftClient.ini` on clean exit, so generate a temp config per run and do not edit it while MCC is still running.
+
+## Auth prompt handling
+
+- Do not send a bare Enter to dismiss `Password(invisible):` or `Paste your code here:` prompts. That can trigger offline fallback.
+- For interactive online-mode runs, wait for the device code prompt and relay the code to the user exactly as shown.
+- After the user completes login, continue the test in the same TTY session or restart into file-driven mode if the workflow requires automation.
+
+## Join-time noise
+
+- Real user configs may contain enabled bots or task lists that were harmless in offline testing but are noisy in online-mode validation.
+- The most common examples are:
+  - `ChatBot.ScriptScheduler` task lists that send `/hello`, `/login ...`, or other automatic commands on login or on an interval
+  - `ChatBot.DiscordRpc`, which is not harmful to server state but adds log noise and extra background activity
+- If the goal is protocol or feature validation, suppress these before the run or treat their output as non-test noise.
+
+## Server settings
+
+- For realistic online-mode testing, keep `online-mode=true`.
+- Keep `enforce-secure-profile=true` unless the test explicitly targets insecure-profile behavior.
+
+## Command reminders
+
+- With `InternalCmdChar = "slash"`:
+  - `/health`, `/pos`, `/inventory`, `/entity` are MCC internal commands.
+  - `/send /list` and `/send /give ...` are server commands.
+  - bare text is regular chat sent to the server.

.skills/mcc-integration-testing/scripts/get_server_port.sh

@@ -0,0 +1,35 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+if [[ $# -ne 1 ]]; then
+    echo "Usage: $0 <server-dir>" >&2
+    exit 1
+fi
+
+REPO_ROOT="$(cd "$(dirname "$0")/../../.." && pwd)"
+SERVER_DIR_NAME="$1"
+SERVERS_ROOT="${MCC_SERVERS:-$REPO_ROOT/MinecraftOfficial/downloads}"
+SERVER_DIR="$SERVERS_ROOT/$SERVER_DIR_NAME"
+PROPS_FILE="$SERVER_DIR/server.properties"
+LATEST_LOG="$SERVER_DIR/logs/latest.log"
+
+if [[ -f "$PROPS_FILE" ]]; then
+    PORT_LINE="$(grep -E '^server-port=' "$PROPS_FILE" | tail -n 1 || true)"
+    if [[ -n "$PORT_LINE" ]]; then
+        PORT="${PORT_LINE#server-port=}"
+        if [[ "$PORT" =~ ^[0-9]+$ ]]; then
+            printf '%s\n' "$PORT"
+            exit 0
+        fi
+    fi
+fi
+
+if [[ -f "$LATEST_LOG" ]]; then
+    PORT="$(sed -n 's/.*Starting Minecraft server on .*:\([0-9][0-9]*\).*/\1/p' "$LATEST_LOG" | tail -n 1)"
+    if [[ -n "$PORT" ]]; then
+        printf '%s\n' "$PORT"
+        exit 0
+    fi
+fi
+
+printf '25565\n'

.skills/mcc-integration-testing/scripts/prepare_offline_mcc_config.sh

@@ -0,0 +1,53 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+if [[ $# -lt 3 || $# -gt 4 ]]; then
+    echo "Usage: $0 <template-ini> <output-ini> <mc-version> [login]" >&2
+    exit 1
+fi
+
+TEMPLATE_INI="$1"
+OUTPUT_INI="$2"
+MC_VERSION="$3"
+LOGIN_NAME="${4:-CursorBot}"
+ACCOUNT_TYPE="${MCC_TEST_ACCOUNT_TYPE:-mojang}"
+PASSWORD_VALUE="${MCC_TEST_PASSWORD-}"
+
+if [[ "$ACCOUNT_TYPE" != "mojang" && "$ACCOUNT_TYPE" != "microsoft" && "$ACCOUNT_TYPE" != "yggdrasil" ]]; then
+    echo "Unsupported MCC_TEST_ACCOUNT_TYPE: $ACCOUNT_TYPE" >&2
+    exit 1
+fi
+
+if [[ -z "${MCC_TEST_PASSWORD+x}" ]]; then
+    if [[ "$ACCOUNT_TYPE" == "mojang" ]]; then
+        PASSWORD_VALUE="-"
+    else
+        PASSWORD_VALUE=""
+    fi
+fi
+
+cp "$TEMPLATE_INI" "$OUTPUT_INI"
+
+sed -i \
+    -e "s#^Account = .*#Account = { Login = \"$LOGIN_NAME\", Password = \"$PASSWORD_VALUE\" }#" \
+    -e "s#^AccountType = .*#AccountType = \"$ACCOUNT_TYPE\"#" \
+    -e "s#^MinecraftVersion = \"[^\"]*\"\\(.*\\)\$#MinecraftVersion = \"$MC_VERSION\"\\1#" \
+    -e 's#^TerrainAndMovements = false#TerrainAndMovements = true#' \
+    -e 's#^InventoryHandling = false#InventoryHandling = true#' \
+    -e 's#^EntityHandling = false#EntityHandling = true#' \
+    -e 's#^AutoRespawn = false#AutoRespawn = true#' \
+    "$OUTPUT_INI"
+
+grep -Fq "AccountType = \"$ACCOUNT_TYPE\"" "$OUTPUT_INI" || {
+    echo "Failed to enforce account type $ACCOUNT_TYPE in $OUTPUT_INI" >&2
+    exit 1
+}
+
+if [[ "$ACCOUNT_TYPE" == "mojang" ]]; then
+    grep -Eq '^Account = \{ Login = ".*", Password = "-" \}' "$OUTPUT_INI" || {
+        echo "Failed to enforce offline account in $OUTPUT_INI" >&2
+        exit 1
+    }
+fi
+
+printf '%s\n' "$OUTPUT_INI"

.skills/mcc-integration-testing/scripts/run_full_spectrum_test.sh

@@ -38,24 +38,7 @@ cleanup() {
 trap cleanup EXIT
 
 prepare_config() {
-    cp "$REPO_ROOT/MinecraftClient.ini" "$CFG"
-
-    sed -i \
-        -e 's/Account = { Login = "test", Password = "-" }/Account = { Login = "CursorBot", Password = "-" }/' \
-        -e "s/MinecraftVersion = \"auto\"/MinecraftVersion = \"$MC_VERSION\"/" \
-        -e 's/TerrainAndMovements = false/TerrainAndMovements = true/' \
-        -e 's/InventoryHandling = false/InventoryHandling = true/' \
-        -e 's/EntityHandling = false/EntityHandling = true/' \
-        -e 's/AutoRespawn = false/AutoRespawn = true/' \
-        "$CFG"
-
-    sed -i '/^\[ChatBot.ScriptScheduler\]/,/^\[/ { s/^Enabled = true/Enabled = false/; }' "$CFG"
-    sed -i '/^\[ChatBot.DiscordRpc\]/,/^\[/ { s/^Enabled = true/Enabled = false/; }' "$CFG"
-    sed -i '/^\[ChatBot.AntiAFK\]/,/^\[/ { s/^Enabled = true/Enabled = false/; }' "$CFG"
-    sed -i '/^\[ChatBot.AutoDig\]/,/^\[/ { s/^Enabled = true/Enabled = false/; }' "$CFG"
-    sed -i '/^\[ChatBot.AutoAttack\]/,/^\[/ { s/^Enabled = true/Enabled = false/; }' "$CFG"
-    sed -i '/^\[ChatBot.PlayerListLogger\]/,/^\[/ { s/^Enabled = true/Enabled = false/; }' "$CFG"
-    sed -i '/^\[ChatBot.ReplayCapture\]/,/^\[/ { s/^Enabled = true/Enabled = false/; }' "$CFG"
+    bash "$SCRIPT_DIR/prepare_offline_mcc_config.sh" "$REPO_ROOT/MinecraftClient.ini" "$CFG" "$MC_VERSION" >/dev/null
 }
 
 wait_for_file_pattern() {
@@ -165,6 +148,7 @@ run_mcc_command() {
 
 "$SCRIPT_DIR/ensure_offline_server.sh" "$VERSION"
 prepare_config
+SERVER_PORT="$(bash "$SCRIPT_DIR/get_server_port.sh" "$VERSION")"
 
 : > "$INPUT_FILE"
 
@@ -178,7 +162,7 @@ wait_for_server_ready || fail "Server did not become ready"
 echo "Starting MCC..."
 (
     cd "$REPO_ROOT"
-    MCC_FILE_INPUT=1 dotnet run --project MinecraftClient -c Release --no-build -- "$CFG" > "$MCC_LOG" 2>&1
+    MCC_FILE_INPUT=1 dotnet run --project MinecraftClient -c Release --no-build -- "$CFG" CursorBot - "localhost:$SERVER_PORT" > "$MCC_LOG" 2>&1
 ) &
 MCC_PID=$!