diff --git a/.cursor/skills/mcc-version-adaptation/SKILL.md b/.cursor/skills/mcc-version-adaptation/SKILL.md index b7f26464..44023b06 100644 --- a/.cursor/skills/mcc-version-adaptation/SKILL.md +++ b/.cursor/skills/mcc-version-adaptation/SKILL.md @@ -16,6 +16,38 @@ Systematic workflow for updating Minecraft Console Client to support a new Minec java -jar MinecraftDecompiler.jar --version --side SERVER \ --decompile --output -remapped.jar --decompiled-output -decompiled ``` +- A test server of the target version in `$MCC_SERVERS/` (see `mcc-dev-workflow` skill) + +## Step 0: Generate Server Reports (CRITICAL since 1.21.9) + +**Before** analyzing decompiled source, generate authoritative registry data from the server jar: + +```bash +cd /tmp && java -DbundlerMainClass=net.minecraft.data.Main \ + -jar $MCC_SERVERS/-Vanilla/server.jar \ + --reports --output /tmp/mc_reports +``` + +This produces `/tmp/mc_reports/reports/` containing: +- `registries.json` — all registries with **actual protocol_id** for each entry +- `blocks.json` — all blocks with **block state IDs** +- `packets.json` — packet protocol definitions + +**Why this matters**: Since MC 1.21.9, some items and blocks are registered outside `Items.java`/`Blocks.java` field declarations (via block registration callbacks or other paths). The decompiled source alone will **miss** these entries. The server data generator is the only authoritative source for protocol IDs. + +### Validation check +Compare server registry counts against decompiled source counts: +```bash +python3 -c " +import json +with open('/tmp/mc_reports/reports/registries.json') as f: + data = json.load(f) +for reg in ['minecraft:item', 'minecraft:entity_type', 'minecraft:block']: + print(f'{reg}: {len(data[reg][\"entries\"])} entries') +" +``` + +If server counts differ from decompiled Java source counts, the palette **must** be generated from server data, not from Java source. ## Step 1: Run Registry Diff @@ -33,18 +65,50 @@ This compares five registries and reports which need palette updates: | DataComponents.java | `StructuredComponents/StructuredComponentsRegistryXXX.cs` | New/reordered components | | EntityDataSerializers.java | `EntityMetadataPalettes/EntityMetadataPaletteXXX.cs` | New/reordered serializer types | +**Important**: diff_registries.py compares decompiled Java source. If Step 0 revealed count mismatches, the diff output may undercount. Always cross-reference with server registries.json. + ## Step 2: Generate Updated Palettes For registries marked "PALETTE UPDATE NEEDED": ### Item Palette + +**Preferred method** (accurate since 1.21.9): +```bash +python3 $MCC_REPO/tools/gen_item_palette.py --from-registry /tmp/mc_reports/reports/registries.json +# e.g., gen_item_palette.py --from-registry /tmp/mc_reports/reports/registries.json 1219 +``` + +**Legacy method** (works for versions where Items.java has all items): ```bash python3 $MCC_REPO/tools/gen_item_palette.py # e.g., gen_item_palette.py 1.21.1 121 ``` + - If new items are reported missing from `ItemType.cs`, add them to the enum in alphabetical order. - The script auto-generates the C# palette file. +### Block Palette + +**Preferred method** (accurate since 1.21.9): +```bash +python3 $MCC_REPO/tools/gen_block_palette.py /tmp/mc_reports/reports/blocks.json +# e.g., gen_block_palette.py /tmp/mc_reports/reports/blocks.json 1219 +``` + +**Legacy method** (manual creation from decompiled Blocks.java): Follow the pattern of existing palette files, using `register("name", ...)` call order from the decompiled source. Only reliable when Blocks.java contains all blocks. + +If new blocks are reported missing from `Material.cs`, add them to the enum in alphabetical order. + +### Entity Palette + +```bash +python3 $MCC_REPO/tools/gen_entity_palette.py /tmp/mc_reports/reports/registries.json +# e.g., gen_entity_palette.py /tmp/mc_reports/reports/registries.json 1219 +``` + +If new entity types are reported missing from `EntityType.cs`, add them to the enum in alphabetical order. + ### Entity Metadata Palette ```bash python3 $MCC_REPO/tools/gen_entity_metadata_palette.py @@ -55,9 +119,6 @@ python3 $MCC_REPO/tools/gen_entity_metadata_palette.py 2. MCC's `EntityMetaDataType.cs` enum 3. `DataTypes.cs` read logic (add a `case` to consume the correct bytes) -### Entity/Block Palettes -No generator script yet — these change rarely. When needed, manually create by following the pattern of existing palette files, using `register("name", ...)` call order from the decompiled source. - ### DataComponents / StructuredComponents Compare `DataComponents.java` registration order. If new components appear, update `StructuredComponentsRegistryXXX.cs`. For new component types, implement corresponding reader in `StructuredComponents/Components/`. @@ -72,10 +133,31 @@ After creating palette files, update version selection logic: | Block | `Protocol18.cs` → `blockPalette` initialization | | EntityMetadata | `EntityMetadataPalette.cs` → `GetPalette()` switch | | DataComponents | `StructuredComponentsRegistry.cs` → factory/routing | +| Packet | `PacketType18Handler.cs` → `GetTypeHandler()` switch | Pattern: add a new `>= MC_X_Y_Z_Version => new XxxPaletteXYZ()` case. -## Step 4: Check Variant Encoding Changes +Also update: +- `Protocol18.cs`: add `MC_X_Y_Z_Version = ` constant +- `Protocol18.cs`: update all `> MC_prev_Version` upper-bound checks to `> MC_X_Y_Z_Version` +- `ProtocolHandler.cs`: add version string → protocol mapping, protocol → version mapping, add to supported list +- `Program.cs`: update `MCHighestVersion` + +## Step 4: Check Packet Changes + +Compare `GameProtocols.java` and `ConfigurationProtocols.java` between versions. + +Common patterns: +- **New clientbound packets inserted mid-list**: All subsequent packet IDs shift. Requires a new `PacketPalette` class. +- **New packets appended at end**: Only need to add new enum values and entries in the palette. +- **Packet renames** (same slot): Update MCC's packet type enum name but no ID change. + +When packet changes are detected: +1. Add new packet type enum values to `PacketTypesIn.cs`, `PacketTypesOut.cs`, `ConfigurationPacketTypesIn.cs`, `ConfigurationPacketTypesOut.cs` +2. Create new `PacketPaletteXXX.cs` based on the previous one, adjusting IDs +3. Update `PacketType18Handler.cs` routing + +## Step 5: Check Variant Encoding Changes For entity types that use variant serializers (Cat, Wolf, Frog, Painting), check if the codec changed between versions by inspecting: @@ -85,18 +167,26 @@ For entity types that use variant serializers (Cat, Wolf, Frog, Painting), check - `ByteBufCodecs.holder()` → wire format: `VarInt(id+1)` for registered, `VarInt(0) + inline_data` for direct - If codec changed, update `DataTypes.cs` entity metadata reading logic accordingly. -## Step 5: Handle New EntityDataSerializer Types +## Step 6: Handle New EntityDataSerializer Types When new serializer types are added (detected in Step 1): 1. Add enum value to `EntityMetaDataType.cs` with XML doc comment 2. Add read logic in `DataTypes.cs` `ReadNextMetadata()`: - Determine byte consumption from the decompiled codec - - Examples: VarInt read, list of particles, etc. + - Simple enum types (like CopperGolemState, WeatheringCopperState): `ReadNextVarInt(cache)` + - Composite types (like ResolvableProfile): analyze the STREAM_CODEC chain in decompiled source 3. Create the new palette file (Step 2) 4. Update palette routing (Step 3) -## Step 6: Compile and Verify +## Step 7: Check SpawnEntity / Other Packet Format Changes + +Compare key packet codec classes between versions. Known changes: +- **1.21.9+**: `SpawnEntity` velocity fields changed from `short / 8000.0` to `LpVec3` format (VarLong-packed fixed-point). Gate reading in `DataTypes.ReadNextEntity()` by version. + +When in doubt, compare the relevant packet class (e.g. `ClientboundAddEntityPacket.java`) between versions. + +## Step 8: Compile and Verify ```bash dotnet build $MCC_REPO/MinecraftClient.sln -c Release @@ -104,27 +194,53 @@ dotnet build $MCC_REPO/MinecraftClient.sln -c Release Then connect to a test server of the target version (see `mcc-dev-workflow` skill) and verify: - Successful connection -- `/give` new items → check inventory -- Summon entities (especially variant types) → no metadata parse errors -- Particle effects → no crashes +- `/give` new items → check inventory for correct identification +- `/give` existing items (diamond_sword, etc.) → verify no ID shift +- Summon new entities → check type and health +- Summon variant entities (wolf, cat, frog) → no metadata parse errors +- Place new blocks → `dig` reports correct block type +- Teleport to distant chunks → terrain loads without errors +- Chat commands work normally + +**Always verify basic existing items first** (e.g. diamond_sword) to catch palette ID shift bugs early. If an existing item shows as the wrong type, the palette is using wrong protocol IDs. ## Key Source Files Reference | Decompiled Java Source | Purpose | |----------------------|---------| -| `world/item/Items.java` | Item registry (field declaration order = ID) | +| `world/item/Items.java` | Item registry (field declaration order ≈ ID, **but not always since 1.21.9**) | | `world/entity/EntityType.java` | Entity type registry (`register()` call order = ID) | -| `world/level/block/Blocks.java` | Block registry (`register()` call order = ID) | +| `world/level/block/Blocks.java` | Block registry (`register()` call order ≈ ID, **but not always since 1.21.9**) | | `core/component/DataComponents.java` | Data component registry | | `network/syncher/EntityDataSerializers.java` | Entity metadata type registry (static block order = ID) | +| `network/protocol/game/GameProtocols.java` | Play packet registration order (= packet IDs) | +| `network/protocol/configuration/ConfigurationProtocols.java` | Config packet registration order | + +| Server Data Generator Output | Purpose | +|-----|---------| +| `registries.json` | **Authoritative** protocol_id for all registries | +| `blocks.json` | **Authoritative** block state IDs | +| `packets.json` | Packet protocol definitions | ## Common Pitfalls -- **ID order matters**: IDs are determined by declaration/registration order, not alphabetical. Always use the decompiled source as ground truth. +- **Source field order ≠ runtime registry ID (since 1.21.9)**: Some items/blocks are registered via callbacks (e.g., block items registered by `Blocks.java` during block registration) rather than in `Items.java` field declarations. Always validate palette counts against server `registries.json`. If counts differ, **use server data generator output instead of decompiled source**. +- **ID order matters**: IDs are determined by registration order, not alphabetical. Always use server data generator as ground truth. - **Cross-version jumps**: When MCC skips versions (e.g., 1.20.4→1.20.6), registries from ALL intermediate versions may have changed. Always diff against the actual last-supported version, not the latest palette. - **EntityMetadata type shifts**: A single new serializer type shifts all subsequent IDs, causing widespread metadata parse failures. Symptoms: entity rendering glitches, disconnections, or silent data corruption. - **CUT_STANDSTONE_SLAB**: This is an intentional typo in Minecraft source (should be SANDSTONE). MCC's `ItemType.cs` uses `CutSandstoneSlab` — the gen script handles this via the OVERRIDES dict. +- **Item/block renames across versions**: Some items/blocks get renamed (e.g., `DRY_SHORT_GRASS` → `SHORT_DRY_GRASS`, `CHAIN` → `IRON_CHAIN`). Keep old enum values for backward compatibility with older palettes, and add new ones for the new version. +- **Packet ID cascading shifts**: Even one inserted mid-list clientbound packet shifts ALL subsequent IDs. Always create a new PacketPalette for protocol changes. +- **Test existing items first**: After palette changes, always verify existing items (diamond_sword, stone, etc.) before testing new ones. If they show as wrong items, the palette has a systemic ID offset bug. ## Reusable Scripts All scripts are in `$MCC_REPO/tools/`. See `tools/README.md` for detailed usage. + +| Script | Purpose | Input | +|--------|---------|-------| +| `diff_registries.py` | Compare registries between versions | Decompiled source | +| `gen_item_palette.py` | Generate ItemPalette C# | Decompiled source OR registries.json | +| `gen_block_palette.py` | Generate BlockPalette C# | blocks.json | +| `gen_entity_palette.py` | Generate EntityPalette C# | registries.json | +| `gen_entity_metadata_palette.py` | Generate EntityMetadataPalette C# | Decompiled source | diff --git a/tools/README.md b/tools/README.md index cce914df..17df9676 100644 --- a/tools/README.md +++ b/tools/README.md @@ -2,46 +2,115 @@ Scripts for analyzing Minecraft version differences and generating MCC palette files. -Requires: Python 3.10+, decompiled MC server source in `MinecraftOfficial/-decompiled/`. +Requires: Python 3.10+ -## Decompiling a new MC version +## Data Sources + +Two types of data can be used as input: + +| Source | How to Get | Authoritative? | +|--------|-----------|---------------| +| Decompiled Java source | `MinecraftDecompiler.jar` → `MinecraftOfficial/-decompiled/` | Mostly (see caveat below) | +| Server data reports | `java -DbundlerMainClass=net.minecraft.data.Main -jar server.jar --reports` | **Yes** | + +**Important since MC 1.21.9**: Some items and blocks are registered outside `Items.java`/`Blocks.java` field declarations (via block registration callbacks). In these cases, the decompiled source undercounts entries. **Always use server data reports** for item and block palettes when available. + +### Decompiling a new MC version ```bash cd MinecraftOfficial -java -jar MinecraftDecompiler.jar --version 1.21.4 --side SERVER \ - --decompile --output 1.21.4-remapped.jar --decompiled-output 1.21.4-decompiled +java -jar MinecraftDecompiler.jar --version 1.21.9 --side SERVER \ + --decompile --output 1.21.9-remapped.jar --decompiled-output 1.21.9-decompiled ``` +### Generating server data reports + +```bash +cd /tmp +java -DbundlerMainClass=net.minecraft.data.Main \ + -jar /path/to/server.jar \ + --reports --output /tmp/mc_reports +``` + +This generates: +- `/tmp/mc_reports/reports/registries.json` — all registries with protocol IDs +- `/tmp/mc_reports/reports/blocks.json` — all blocks with block state IDs +- `/tmp/mc_reports/reports/packets.json` — packet protocol definitions + ## diff_registries.py — Compare registries between versions Compares Items, EntityTypes, Blocks, DataComponents, and EntityDataSerializers between two MC versions. Reports whether each palette needs updating, lists added/removed entries, and shows ID shift statistics. ```bash -python3 tools/diff_registries.py 1.20.6 1.21.1 +# Basic comparison (decompiled source only) +python3 tools/diff_registries.py 1.21.8 1.21.9 + +# With cross-validation against server registries.json (recommended) +python3 tools/diff_registries.py 1.21.8 1.21.9 --registry /tmp/mc_reports/reports/registries.json ``` +The `--registry` flag enables cross-validation: compares the count and set of entries found in decompiled Java source against the server's authoritative registry. Any mismatches indicate that palette generation must use server data instead of Java source. + Output indicates for each registry: - **IDENTICAL** → reuse existing palette - **PALETTE UPDATE NEEDED** → create new palette file + update version routing +- **Count MISMATCH** (with --registry) → server has entries not in Java source ## gen_item_palette.py — Generate ItemPalette C# file -Reads `Items.java` field declaration order to generate a complete `ItemPaletteXXX.cs`. +Two modes: ```bash +# Preferred: from server registries.json (accurate since 1.21.9) +python3 tools/gen_item_palette.py --from-registry /tmp/mc_reports/reports/registries.json 1219 + +# Legacy: from decompiled Items.java python3 tools/gen_item_palette.py 1.21.1 121 -# → MinecraftClient/Inventory/ItemPalettes/ItemPalette121.cs ``` -Also validates each item name against `ItemType.cs` and warns about missing enum values. +Output: `MinecraftClient/Inventory/ItemPalettes/ItemPalette.cs` + +Validates each item name against `ItemType.cs` and warns about missing enum values. Add missing values to `ItemType.cs` in alphabetical order before compiling. + +## gen_block_palette.py — Generate BlockPalette C# file + +```bash +python3 tools/gen_block_palette.py /tmp/mc_reports/reports/blocks.json 1219 +# → MinecraftClient/Mapping/BlockPalettes/Palette1219.cs +``` + +Generates a complete block palette with block state ID ranges from the server's `blocks.json`. Validates against `Material.cs` and warns about missing enum values. + +## gen_entity_palette.py — Generate EntityPalette C# file + +```bash +python3 tools/gen_entity_palette.py /tmp/mc_reports/reports/registries.json 1219 +# → MinecraftClient/Mapping/EntityPalettes/EntityPalette1219.cs +``` + +Generates entity type palette from server's `registries.json`. Validates against `EntityType.cs` and warns about missing enum values. ## gen_entity_metadata_palette.py — Generate EntityMetadataPalette C# file -Reads `EntityDataSerializers.java` static block registration order to generate `EntityMetadataPaletteXXX.cs`. - ```bash -python3 tools/gen_entity_metadata_palette.py 1.20.6 1206 -# → MinecraftClient/Mapping/EntityMetadataPalettes/EntityMetadataPalette1206.cs +python3 tools/gen_entity_metadata_palette.py 1.21.9 1219 +# → MinecraftClient/Mapping/EntityMetadataPalettes/EntityMetadataPalette1219.cs ``` -The script maps Java field names to MCC's `EntityMetaDataType` enum. If a new serializer type appears that isn't in the mapping table, it will warn you to update both the script's `FIELD_TO_ENUM` dict and MCC's `EntityMetaDataType.cs` enum. +Reads `EntityDataSerializers.java` static block registration order. Maps Java field names to MCC's `EntityMetaDataType` enum. If a new serializer type appears that isn't in the mapping table, it will warn you to update: +1. The script's `FIELD_TO_ENUM` dict +2. MCC's `EntityMetaDataType.cs` enum +3. `DataTypes.cs` ReadNextMetadata() read logic + +## Recommended workflow + +1. Generate server reports (Step 0) +2. Run `diff_registries.py --registry` to identify changes and validate source completeness +3. For each registry needing update: + - Items: `gen_item_palette.py --from-registry` + - Blocks: `gen_block_palette.py` + - Entities: `gen_entity_palette.py` + - Metadata: `gen_entity_metadata_palette.py` +4. Add any missing enum values to `ItemType.cs`, `Material.cs`, `EntityType.cs`, `EntityMetaDataType.cs` +5. Update version routing (see SKILL.md) +6. Build and test diff --git a/tools/diff_registries.py b/tools/diff_registries.py index cc26c31f..aba04dff 100644 --- a/tools/diff_registries.py +++ b/tools/diff_registries.py @@ -6,12 +6,19 @@ Compares Items, EntityTypes, Blocks, DataComponents, and EntityDataSerializers to determine which MCC palettes need updating for a new MC version. Usage: - python3 tools/diff_registries.py + python3 tools/diff_registries.py [--registry ] -Example: +Examples: python3 tools/diff_registries.py 1.20.6 1.21.1 + python3 tools/diff_registries.py 1.21.8 1.21.9 --registry /tmp/mc_reports/reports/registries.json + +The optional --registry flag cross-validates decompiled source counts against +the server's authoritative registries.json (generated via --reports). +Since MC 1.21.9, some items/blocks are registered outside Items.java/Blocks.java, +making this cross-validation essential for detecting hidden entries. """ +import json import re import sys import os @@ -65,7 +72,7 @@ def extract_static_register_order(filepath: Path) -> list[str]: return results -def compare_lists(old: list[str], new: list[str], label: str): +def compare_lists(old: list[str], new: list[str], label: str) -> bool: """Compare two ordered lists and report differences.""" set_old, set_new = set(old), set(new) added = sorted(set_new - set_old) @@ -117,7 +124,46 @@ def compare_lists(old: list[str], new: list[str], label: str): return True -def diff_items(old_dir: Path, new_dir: Path): +def cross_validate(registry_data: dict, java_entries: list[str], registry_key: str, + label: str, convert_fn=None): + """Cross-validate Java source entries against server registries.json.""" + reg = registry_data.get(registry_key, {}).get("entries", {}) + server_names = set() + for key in reg: + name = key.removeprefix("minecraft:") + server_names.add(name) + + if convert_fn: + java_names = set(convert_fn(n) for n in java_entries) + else: + java_names = set(n.lower() for n in java_entries) + + server_count = len(server_names) + java_count = len(java_names) + + print(f"\n --- Cross-validation: {label} ---") + print(f" Java source: {java_count} entries, Server registry: {server_count} entries") + + if server_count == java_count: + print(f" ✓ Counts match — Java source is complete") + else: + diff = server_count - java_count + print(f" ⚠ Count MISMATCH: server has {diff:+d} entries vs Java source") + extra_in_server = server_names - java_names + extra_in_java = java_names - server_names + if extra_in_server: + print(f" In server but NOT in Java source ({len(extra_in_server)}):") + for n in sorted(extra_in_server): + pid = reg[f"minecraft:{n}"]["protocol_id"] + print(f" [{pid}] {n}") + print(f" ⚠ MUST use --from-registry / server data to generate palette!") + if extra_in_java: + print(f" In Java source but NOT in server ({len(extra_in_java)}):") + for n in sorted(extra_in_java): + print(f" {n}") + + +def diff_items(old_dir: Path, new_dir: Path, registry_data: dict | None = None): old_f = find_java_file(old_dir, "net/minecraft/world/item/Items.java") new_f = find_java_file(new_dir, "net/minecraft/world/item/Items.java") if not old_f or not new_f: @@ -128,8 +174,12 @@ def diff_items(old_dir: Path, new_dir: Path): new = extract_field_names(new_f, pattern) compare_lists(old, new, "Items.java (Item registry)") + if registry_data: + cross_validate(registry_data, new, "minecraft:item", "Items", + convert_fn=lambda n: n.lower()) -def diff_entity_types(old_dir: Path, new_dir: Path): + +def diff_entity_types(old_dir: Path, new_dir: Path, registry_data: dict | None = None): old_f = find_java_file(old_dir, "net/minecraft/world/entity/EntityType.java") new_f = find_java_file(new_dir, "net/minecraft/world/entity/EntityType.java") if not old_f or not new_f: @@ -139,8 +189,12 @@ def diff_entity_types(old_dir: Path, new_dir: Path): new = extract_register_multiline(new_f) compare_lists(old, new, "EntityType.java (Entity registry)") + if registry_data: + cross_validate(registry_data, new, "minecraft:entity_type", "EntityType", + convert_fn=lambda n: n) -def diff_blocks(old_dir: Path, new_dir: Path): + +def diff_blocks(old_dir: Path, new_dir: Path, registry_data: dict | None = None): old_f = find_java_file(old_dir, "net/minecraft/world/level/block/Blocks.java") new_f = find_java_file(new_dir, "net/minecraft/world/level/block/Blocks.java") if not old_f or not new_f: @@ -150,6 +204,10 @@ def diff_blocks(old_dir: Path, new_dir: Path): new = extract_register_multiline(new_f) compare_lists(old, new, "Blocks.java (Block registry)") + if registry_data: + cross_validate(registry_data, new, "minecraft:block", "Blocks", + convert_fn=lambda n: n) + def diff_data_components(old_dir: Path, new_dir: Path): old_f = find_java_file(old_dir, "net/minecraft/core/component/DataComponents.java") @@ -183,11 +241,23 @@ def diff_entity_data_serializers(old_dir: Path, new_dir: Path): def main(): - if len(sys.argv) != 3: + # Parse arguments + args = sys.argv[1:] + registry_path = None + + if "--registry" in args: + idx = args.index("--registry") + if idx + 1 >= len(args): + print("Error: --registry requires a path argument") + sys.exit(1) + registry_path = Path(args[idx + 1]) + args = args[:idx] + args[idx + 2:] + + if len(args) != 2: print(__doc__) sys.exit(1) - old_ver, new_ver = sys.argv[1], sys.argv[2] + old_ver, new_ver = args[0], args[1] old_dir = DECOMPILED_ROOT / f"{old_ver}-decompiled" new_dir = DECOMPILED_ROOT / f"{new_ver}-decompiled" @@ -199,13 +269,22 @@ def main(): f"--output {v}-remapped.jar --decompiled-output {v}-decompiled") sys.exit(1) + registry_data = None + if registry_path: + if not registry_path.exists(): + print(f"Error: {registry_path} not found") + sys.exit(1) + with open(registry_path) as f: + registry_data = json.load(f) + print(f"Loaded server registries.json for cross-validation") + print(f"Comparing MC {old_ver} → {new_ver}") print(f"Old: {old_dir}") print(f"New: {new_dir}") - diff_items(old_dir, new_dir) - diff_entity_types(old_dir, new_dir) - diff_blocks(old_dir, new_dir) + diff_items(old_dir, new_dir, registry_data) + diff_entity_types(old_dir, new_dir, registry_data) + diff_blocks(old_dir, new_dir, registry_data) diff_data_components(old_dir, new_dir) diff_entity_data_serializers(old_dir, new_dir) @@ -215,6 +294,10 @@ def main(): print(" Review each section above. For any marked 'PALETTE UPDATE NEEDED',") print(" create a new palette file in MCC and update the version routing.") print(" For 'IDENTICAL' sections, the existing palette can be reused.") + if registry_data: + print("\n Cross-validation was performed against server registries.json.") + print(" If any count mismatches were found, use server data generator output") + print(" (--from-registry) instead of decompiled Java source for palette generation.") if __name__ == "__main__": diff --git a/tools/gen_block_palette.py b/tools/gen_block_palette.py new file mode 100644 index 00000000..03c48eca --- /dev/null +++ b/tools/gen_block_palette.py @@ -0,0 +1,119 @@ +#!/usr/bin/env python3 +""" +Generate an MCC BlockPalette C# file from server-generated blocks.json. + +The blocks.json file is generated by running: + java -DbundlerMainClass=net.minecraft.data.Main -jar server.jar --reports + +Usage: + python3 tools/gen_block_palette.py + +Example: + python3 tools/gen_block_palette.py /tmp/mc_reports/reports/blocks.json 1219 + # Generates Palette1219.cs +""" + +import json +import re +import sys +from pathlib import Path + +OUTPUT_DIR = (Path(__file__).resolve().parent.parent / + "MinecraftClient" / "Mapping" / "BlockPalettes") +MATERIAL_CS = OUTPUT_DIR.parent / "Material.cs" + + +def mc_name_to_csharp(mc_name: str) -> str: + """Convert minecraft:snake_case to PascalCase C# enum name.""" + name = mc_name.removeprefix("minecraft:") + return "".join(word.capitalize() for word in name.split("_")) + + +def load_known_materials() -> set[str]: + known = set() + if MATERIAL_CS.exists(): + with open(MATERIAL_CS) as f: + for line in f: + m = re.match(r'\s+(\w+),?\s*$', line) + if m: + known.add(m.group(1)) + return known + + +def main(): + if len(sys.argv) != 3: + print(__doc__) + sys.exit(1) + + blocks_json = Path(sys.argv[1]) + class_suffix = sys.argv[2] + + if not blocks_json.exists(): + print(f"Error: {blocks_json} not found") + sys.exit(1) + + with open(blocks_json) as f: + data = json.load(f) + + # Build (min_state, max_state, cs_name) for each block, sorted by min_state + block_ranges = [] + for block_key, block_info in data.items(): + cs_name = mc_name_to_csharp(block_key) + states = block_info.get("states", []) + state_ids = [s["id"] for s in states] + if state_ids: + block_ranges.append((min(state_ids), max(state_ids), cs_name)) + + block_ranges.sort(key=lambda x: x[0]) + print(f"Loaded {len(block_ranges)} blocks from {blocks_json}") + + max_state = max(r[1] for r in block_ranges) + print(f"State ID range: 0 - {max_state}") + + known_materials = load_known_materials() + missing = [cs for _, _, cs in block_ranges if known_materials and cs not in known_materials] + if missing: + print(f"\nWARNING: {len(missing)} blocks not found in Material.cs enum:") + for cs_name in missing: + print(f" {cs_name}") + print("\nYou need to add these to Material.cs before the palette will compile.") + print("Insert them in alphabetical order within the enum.") + + class_name = f"Palette{class_suffix}" + output_path = OUTPUT_DIR / f"{class_name}.cs" + + lines = [ + "using System.Collections.Generic;", + "", + "namespace MinecraftClient.Mapping.BlockPalettes", + "{", + f" public class {class_name} : BlockPalette", + " {", + " private static readonly Dictionary materials = new();", + "", + f" static {class_name}()", + " {", + ] + + for min_s, max_s, cs_name in block_ranges: + lines.append(f" for (int i = {min_s}; i <= {max_s}; i++)") + lines.append(f" materials[i] = Material.{cs_name};") + + lines += [ + " }", + "", + " protected override Dictionary GetDict()", + " {", + " return materials;", + " }", + " }", + "}", + "", + ] + + output_path.write_text("\n".join(lines)) + print(f"Generated {output_path} with {len(block_ranges)} blocks ({max_state + 1} total states)") + + +if __name__ == "__main__": + main() diff --git a/tools/gen_entity_palette.py b/tools/gen_entity_palette.py new file mode 100644 index 00000000..a932d591 --- /dev/null +++ b/tools/gen_entity_palette.py @@ -0,0 +1,111 @@ +#!/usr/bin/env python3 +""" +Generate an MCC EntityPalette C# file from server-generated registries.json. + +The registries.json file is generated by running: + java -DbundlerMainClass=net.minecraft.data.Main -jar server.jar --reports + +Usage: + python3 tools/gen_entity_palette.py + +Example: + python3 tools/gen_entity_palette.py /tmp/mc_reports/reports/registries.json 1219 + # Generates EntityPalette1219.cs +""" + +import json +import re +import sys +from pathlib import Path + +OUTPUT_DIR = (Path(__file__).resolve().parent.parent / + "MinecraftClient" / "Mapping" / "EntityPalettes") +ENTITY_TYPE_CS = OUTPUT_DIR.parent / "EntityType.cs" + + +def mc_name_to_csharp(mc_name: str) -> str: + """Convert minecraft:snake_case to PascalCase C# enum name.""" + name = mc_name.removeprefix("minecraft:") + return "".join(word.capitalize() for word in name.split("_")) + + +def load_known_entity_types() -> set[str]: + known = set() + if ENTITY_TYPE_CS.exists(): + with open(ENTITY_TYPE_CS) as f: + for line in f: + m = re.match(r'\s+(\w+),?\s*$', line) + if m: + known.add(m.group(1)) + return known + + +def main(): + if len(sys.argv) != 3: + print(__doc__) + sys.exit(1) + + registry_path = Path(sys.argv[1]) + class_suffix = sys.argv[2] + + if not registry_path.exists(): + print(f"Error: {registry_path} not found") + sys.exit(1) + + with open(registry_path) as f: + data = json.load(f) + + entities_reg = data.get("minecraft:entity_type", {}).get("entries", {}) + mappings = [] + for entity_key, info in entities_reg.items(): + pid = info["protocol_id"] + cs_name = mc_name_to_csharp(entity_key) + mappings.append((pid, cs_name)) + + mappings.sort(key=lambda x: x[0]) + print(f"Loaded {len(mappings)} entity types from {registry_path}") + + known = load_known_entity_types() + missing = [cs for _, cs in mappings if known and cs not in known] + if missing: + print(f"\nWARNING: {len(missing)} entity types not found in EntityType.cs enum:") + for cs_name in missing: + print(f" {cs_name}") + print("\nYou need to add these to EntityType.cs before the palette will compile.") + print("Insert them in alphabetical order within the enum.") + + class_name = f"EntityPalette{class_suffix}" + output_path = OUTPUT_DIR / f"{class_name}.cs" + + lines = [ + "using System.Collections.Generic;", + "", + "namespace MinecraftClient.Mapping.EntityPalettes", + "{", + f" public class {class_name} : EntityPalette", + " {", + " private static readonly Dictionary mappings = new();", + "", + f" static {class_name}()", + " {", + ] + for pid, cs_name in mappings: + lines.append(f" mappings[{pid}] = EntityType.{cs_name};") + lines += [ + " }", + "", + " protected override Dictionary GetDict()", + " {", + " return mappings;", + " }", + " }", + "}", + "", + ] + + output_path.write_text("\n".join(lines)) + print(f"Generated {output_path} with {len(mappings)} entity types") + + +if __name__ == "__main__": + main() diff --git a/tools/gen_item_palette.py b/tools/gen_item_palette.py index 92643977..5a85963b 100644 --- a/tools/gen_item_palette.py +++ b/tools/gen_item_palette.py @@ -1,36 +1,43 @@ #!/usr/bin/env python3 """ -Generate an MCC ItemPalette C# file from decompiled Items.java. +Generate an MCC ItemPalette C# file. -Reads public static final Item field declarations (which define item IDs -by declaration order) and generates a complete C# palette class. +Supports two input modes: + 1. Server registry (preferred since 1.21.9): + python3 tools/gen_item_palette.py --from-registry /tmp/mc_reports/reports/registries.json -Usage: - python3 tools/gen_item_palette.py + 2. Decompiled Items.java (legacy): + python3 tools/gen_item_palette.py -Example: - python3 tools/gen_item_palette.py 1.21.1 121 - # Generates ItemPalette121.cs +The --from-registry mode uses the server's authoritative protocol_id assignments, +which is required since MC 1.21.9 where some items are registered outside Items.java. -The determines the class name (ItemPalette) and -should match MCC's naming convention (e.g., 121 for 1.21, 1206 for 1.20.6). +The determines the class name (ItemPalette) and should match MCC's +naming convention (e.g., 121 for 1.21, 1219 for 1.21.9). """ +import json import re import sys from pathlib import Path DECOMPILED_ROOT = Path(__file__).resolve().parent.parent / "MinecraftOfficial" OUTPUT_DIR = Path(__file__).resolve().parent.parent / "MinecraftClient" / "Inventory" / "ItemPalettes" +ITEM_TYPE_CS = OUTPUT_DIR.parent / "ItemType.cs" -# Java field name → C# ItemType enum name -# Most conversions are automatic (SCREAMING_SNAKE → PascalCase). -# Add manual overrides here for irregular names. OVERRIDES = { "CUT_STANDSTONE_SLAB": "CutSandstoneSlab", # Mojang typo in source } +def mc_name_to_csharp(mc_name: str) -> str: + """Convert minecraft:snake_case to PascalCase C# enum name.""" + name = mc_name.removeprefix("minecraft:") + if name.upper() in OVERRIDES: + return OVERRIDES[name.upper()] + return "".join(word.capitalize() for word in name.split("_")) + + def java_to_csharp_name(java_name: str) -> str: """Convert SCREAMING_SNAKE_CASE Java field name to PascalCase C# enum name.""" if java_name in OVERRIDES: @@ -38,52 +45,81 @@ def java_to_csharp_name(java_name: str) -> str: return "".join(word.capitalize() for word in java_name.lower().split("_")) -def main(): - if len(sys.argv) != 3: - print(__doc__) - sys.exit(1) +def load_known_enums() -> set[str]: + known = set() + if ITEM_TYPE_CS.exists(): + with open(ITEM_TYPE_CS) as f: + for line in f: + m = re.match(r'\s+(\w+),?\s*$', line) + if m and m.group(1) not in ("Null", "Unknown"): + known.add(m.group(1)) + return known - mc_version = sys.argv[1] - class_suffix = sys.argv[2] + +def items_from_registry(registry_path: Path) -> list[tuple[int, str]]: + """Load items from server registries.json, returns sorted (protocol_id, cs_name) pairs.""" + with open(registry_path) as f: + data = json.load(f) + items_reg = data.get("minecraft:item", {}).get("entries", {}) + result = [] + for item_key, info in items_reg.items(): + pid = info["protocol_id"] + cs_name = mc_name_to_csharp(item_key) + result.append((pid, cs_name)) + result.sort(key=lambda x: x[0]) + return result + + +def items_from_java(mc_version: str) -> list[tuple[int, str]]: + """Load items from decompiled Items.java field declaration order.""" version_dir = DECOMPILED_ROOT / f"{mc_version}-decompiled" items_java = version_dir / "net" / "minecraft" / "world" / "item" / "Items.java" - if not items_java.exists(): print(f"Error: {items_java} not found") sys.exit(1) pattern = re.compile(r'\s+public static final Item (\w+)\s*=') - field_names = [] + result = [] with open(items_java) as f: for line in f: m = pattern.match(line) if m: - field_names.append(m.group(1)) + idx = len(result) + cs_name = java_to_csharp_name(m.group(1)) + result.append((idx, cs_name)) + return result - print(f"Found {len(field_names)} items in MC {mc_version}") - # Verify enum name conversion against existing ItemType.cs - item_type_cs = OUTPUT_DIR.parent / "ItemType.cs" - known_enums = set() - if item_type_cs.exists(): - with open(item_type_cs) as f: - for line in f: - m = re.match(r'\s+(\w+),?\s*$', line) - if m and m.group(1) not in ("Null", "Unknown"): - known_enums.add(m.group(1)) +def main(): + if len(sys.argv) < 3: + print(__doc__) + sys.exit(1) - missing = [] - mappings = [] - for i, name in enumerate(field_names): - cs_name = java_to_csharp_name(name) - mappings.append((i, cs_name)) - if known_enums and cs_name not in known_enums: - missing.append((i, name, cs_name)) + from_registry = sys.argv[1] == "--from-registry" + if from_registry: + if len(sys.argv) != 4: + print("Usage: gen_item_palette.py --from-registry ") + sys.exit(1) + registry_path = Path(sys.argv[2]) + class_suffix = sys.argv[3] + if not registry_path.exists(): + print(f"Error: {registry_path} not found") + sys.exit(1) + mappings = items_from_registry(registry_path) + print(f"Loaded {len(mappings)} items from {registry_path}") + else: + mc_version = sys.argv[1] + class_suffix = sys.argv[2] + mappings = items_from_java(mc_version) + print(f"Found {len(mappings)} items in MC {mc_version} Items.java") + + known_enums = load_known_enums() + missing = [(pid, cs) for pid, cs in mappings if known_enums and cs not in known_enums] if missing: print(f"\nWARNING: {len(missing)} items not found in ItemType.cs enum:") - for idx, java_name, cs_name in missing: - print(f" [{idx}] {java_name} -> {cs_name}") + for pid, cs_name in missing: + print(f" [{pid}] {cs_name}") print("\nYou need to add these to ItemType.cs before the palette will compile.") print("Insert them in alphabetical order within the enum.") @@ -102,8 +138,8 @@ def main(): f" static {class_name}()", " {", ] - for idx, cs_name in mappings: - lines.append(f" mappings[{idx}] = ItemType.{cs_name};") + for pid, cs_name in mappings: + lines.append(f" mappings[{pid}] = ItemType.{cs_name};") lines += [ " }", "",