Skip to content

refactor: migrate runtime and resources to UTF-8 first - #3

Merged
comfuture merged 15 commits into
masterfrom
feat/utf-8
Feb 11, 2026
Merged

refactor: migrate runtime and resources to UTF-8 first#3
comfuture merged 15 commits into
masterfrom
feat/utf-8

Conversation

@comfuture

@comfuture comfuture commented Feb 11, 2026

Copy link
Copy Markdown
Owner

Summary

This PR migrates the project to a UTF-8 first model across resources, runtime input handling, string logic, tests, and docs.

Why

The previous implementation depended on CP949 assumptions (2-byte Hangul handling, CP949 smoke setup, byte-based suffix/length logic), which caused correctness and portability problems on modern UTF-8 environments.

Changes

1) Resource and text normalization

  • Normalize legacy byte-path resources to UTF-8-safe tree/manifest handling.
  • Convert CP949 text assets and source text files to UTF-8 (binary artifacts excluded).

2) UTF-8 utility layer

  • Add /Users/comfuture/Project/muhan/src/utf8_text.c and /Users/comfuture/Project/muhan/src/utf8_text.h.
  • Provide helpers for:
    • UTF-8 validation/decoding
    • codepoint length counting
    • UTF-8 suffix checks
    • previous codepoint boundary (backspace support)
    • Hangul jongseong detection

3) Unicode name + player path refactor

  • Replace first_han() path fanout usage with SHA-1 2-char sharding path generation.
  • Add Unicode-aware player-name validation with forbidden character checks.

4) Remove CP949 2-byte assumptions

  • Replace byte-pair logic in command/string routines with Unicode codepoint-based logic.
  • Replace byte-tail suffix checks like len-2 with UTF-8 suffix helpers.

5) Strict UTF-8 socket input pipeline

  • Refactor /Users/comfuture/Project/muhan/src/io.c accept_input() to strict UTF-8 handling.
  • Ignore invalid UTF-8 sequences safely.
  • Handle backspace by codepoint rather than byte.
  • Protect command buffer integrity under malformed input.

6) Smoke/tests and docs

  • Update smoke encoding flow to UTF-8 and add Unicode edge cases (invalid UTF-8 input + multibyte backspace/session survival).
  • Document UTF-8 first policies and compatibility constraints in README/revive docs.

Compatibility notes

  • CP949 player data/input backward compatibility is intentionally not provided.
  • Text files are UTF-8; binary artifacts remain unchanged.

Validation

  • Ran ./scripts/run-smoke.sh after each atomic stage and again on final HEAD.
  • Final smoke result: pass (c-only, rust-resolver parity/session smoke, rust unit tests).

Atomic commits included

  • 93cd8ac chore: normalize legacy byte-path resources to utf8-safe tree
  • 86c3f23 chore: convert cp949 text assets and source files to utf-8
  • 7a0c43b feat: add utf8 utility layer for validation and codepoint operations
  • 3149e4b refactor: support unicode player names and sha1-sharded player paths
  • 8030273 refactor: replace cp949 byte-based string logic with unicode codepoint logic
  • 2dea390 refactor: enforce strict utf-8 handling in socket input pipeline
  • b59f934 test: migrate smoke scenarios to utf-8 and add unicode edge cases
  • b3ff282 docs: document utf-8 first runtime and migration constraints
  • e6d49ed fix: stabilize utf-8 smoke checks on mac and arm runners

Latest CI stabilization update

  • f444a60 fix: make smoke scenario resilient across ci runners
    • Keep UTF-8 edge raw-input probe behind SMOKE_UTF8_EDGE opt-in.
    • Replace brittle hint-content assertions with minimal readable-output gate.
    • Keep reconnect/session progression checks while reducing false negatives.

Runtime safety fix after password-input report

  • 5b5dde2 fix: ensure password null termination in character creation
    • In /Users/comfuture/Project/muhan/src/command1.c, character-creation password copy now guarantees NUL termination.
    • Prevents undefined behavior when password length is exactly 14 bytes.

Runtime crash fix for nc/telnet character creation path

  • c4a698b fix: prevent server crash on character init room load failures
    • /Users/comfuture/Project/muhan/src/player.c
      • init_ply() now returns success/failure and safely handles fallback room load failure without NULL dereference.
    • /Users/comfuture/Project/muhan/src/command1.c
      • login/create flows now check init_ply() result and terminate session safely with an explicit error message when initialization fails.
    • /Users/comfuture/Project/muhan/src/files1.c
      • Added defensive bounds checks for nested object counts, room exit/monster/item counts, and room description length fields.
      • Prevents oversized/negative length-driven allocations from malformed/corrupted room data.
    • /Users/comfuture/Project/muhan/src/mextern.h
      • Updated init_ply declaration to return int.
    • Validation:
      • Reproduced crash with nc on password submission before fix.
      • Confirmed no server process termination after fix (MUHAN_HOME set/unset).
      • Re-ran ./scripts/run-smoke.sh successfully.

Latest runtime fix (init error + nc disconnect symptom)

  • 14dcba3 fix: recover login flow on init failure and tolerate legacy room blobs
    • /Users/comfuture/Project/muhan/src/command1.c
      • On init_ply() failure (both login/create paths), do not hard-disconnect.
      • Clean up player object, reset state, and return to name prompt.
    • /Users/comfuture/Project/muhan/src/player.c
      • Move room load validation before join broadcast/logical side effects.
    • /Users/comfuture/Project/muhan/src/files1.c
      • Keep room loader defensive limits, but treat invalid legacy room count fields as empty sections instead of fatal load error.
    • Validation:
      • Reproduced password-stage "서버 초기화 중 오류가 발생했습니다." path.
      • Confirmed socket remains alive and prompt returns.
      • Confirmed character creation proceeds past password with no init error.
      • Re-ran ./scripts/run-smoke.sh successfully.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b3ff282413

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/command1.c Outdated
Comment thread src/command2.c Outdated
@gemini-code-assist

Copy link
Copy Markdown

Summary of Changes

Hello @comfuture, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

이 풀 리퀘스트는 프로젝트를 UTF-8 우선 모델로 전환하여 정확성과 이식성을 향상시키는 것을 목표로 합니다. 기존 CP949 인코딩에 대한 가정을 제거하고, 리소스 및 텍스트를 UTF-8로 정규화하며, 새로운 UTF-8 유틸리티를 도입하고, 플레이어 이름 및 경로 처리 방식을 유니코드 기반으로 변경하며, 소켓 입력 파이프라인에 엄격한 UTF-8 처리를 적용합니다. 또한, 관련 문서와 테스트 시나리오도 업데이트되었습니다.

Highlights

  • 리소스 및 텍스트 정규화: 레거시 바이트 경로 리소스를 UTF-8 안전 트리/매니페스트 처리로 정규화하고, CP949 텍스트 자산 및 소스 텍스트 파일을 UTF-8로 변환했습니다.
  • UTF-8 유틸리티 계층 추가: UTF-8 유효성 검사/디코딩, 코드포인트 길이 계산, UTF-8 접미사 확인, 이전 코드포인트 경계(백스페이스 지원), 한글 종성 감지를 위한 헬퍼를 제공하는 utf8_text.cutf8_text.h 파일을 추가했습니다.
  • 유니코드 이름 및 플레이어 경로 리팩토링: first_han() 경로 분기 사용을 SHA-1 2문자 샤딩 경로 생성으로 대체하고, 금지 문자 검사를 포함한 유니코드 인식 플레이어 이름 유효성 검사를 추가했습니다.
  • CP949 2바이트 가정 제거: 명령/문자열 루틴의 바이트 쌍 로직을 유니코드 코드포인트 기반 로직으로 교체하고, len-2와 같은 바이트 테일 접미사 검사를 UTF-8 접미사 헬퍼로 대체했습니다.
  • 엄격한 UTF-8 소켓 입력 파이프라인: io.caccept_input() 함수를 엄격한 UTF-8 처리로 리팩토링하여, 유효하지 않은 UTF-8 시퀀스를 안전하게 무시하고, 백스페이스를 바이트 단위가 아닌 코드포인트 단위로 처리하며, 잘못된 입력으로부터 명령 버퍼 무결성을 보호합니다.
  • 테스트 및 문서 업데이트: 스모크 인코딩 흐름을 UTF-8로 업데이트하고 유니코드 엣지 케이스(유효하지 않은 UTF-8 입력 + 멀티바이트 백스페이스/세션 유지)를 추가했으며, README 및 revive 문서에 UTF-8 우선 정책 및 호환성 제약 사항을 문서화했습니다.
Changelog
  • 86c3f23
    • docs/board_make
    • docs/cmd_ital
    • docs/crt_expg
    • docs/crt_flag
    • docs/crt_make
    • docs/crt_talk
    • docs/dm.doc
    • docs/obj_flag
    • docs/obj_make
    • docs/rom_flag
    • docs/rom_make
    • docs/rom_xflg
    • help/char
    • help/exp
    • help/help.1
    • help/help.10
    • help/help.100
    • help/help.101
    • help/help.102
    • help/help.103
    • help/help.104
    • help/help.105
    • help/help.106
    • help/help.107
    • help/help.108
    • help/help.109
    • help/help.11
    • help/help.110
    • help/help.111
    • help/help.112
    • help/help.113
    • help/help.114
    • help/help.115
    • help/help.116
    • help/help.117
    • help/help.118
    • help/help.119
    • help/help.12
    • help/help.120
    • help/help.121
    • help/help.122
    • help/help.123
    • help/help.124
    • help/help.125
    • help/help.126
    • help/help.127
    • help/help.128
    • help/help.130
    • help/help.131
    • help/help.132
    • help/help.133
    • help/help.134
    • help/help.135
    • help/help.136
    • help/help.14
    • help/help.140
    • help/help.141
    • help/help.142
    • help/help.144
    • help/help.145
    • help/help.146
    • help/help.148
    • help/help.149
    • help/help.15
    • help/help.150
    • help/help.151
    • help/help.16
    • help/help.17
    • help/help.18
    • help/help.19
    • help/help.2
    • help/help.20
    • help/help.21
    • help/help.22
    • help/help.23
    • help/help.24
    • help/help.25
    • help/help.26
    • help/help.27
    • help/help.28
    • help/help.29
    • help/help.3
    • help/help.30
    • help/help.32
    • help/help.33
    • help/help.34
    • help/help.35
    • help/help.36
    • help/help.37
    • help/help.38
    • help/help.39
    • help/help.4
    • help/help.40
    • help/help.41
    • help/help.42
    • help/help.43
    • help/help.44
    • help/help.45
    • help/help.46
    • help/help.47
    • help/help.48
    • help/help.49
    • help/help.5
    • help/help.50
    • help/help.51
    • help/help.52
    • help/help.53
    • help/help.54
    • help/help.55
    • help/help.56
    • help/help.57
    • help/help.58
    • help/help.59
    • help/help.6
    • help/help.60
    • help/help.61
    • help/help.62
    • help/help.63
    • help/help.64
    • help/help.65
    • help/help.66
    • help/help.67
    • help/help.68
    • help/help.69
    • help/help.7
    • help/help.70
    • help/help.71
    • help/help.72
    • help/help.73
    • help/help.74
    • help/help.75
    • help/help.76
    • help/help.77
    • help/help.79
    • help/help.8
    • help/help.80
    • help/help.81
    • help/help.82
    • help/help.83
    • help/help.84
    • help/help.85
    • help/help.86
    • help/help.87
    • help/help.88
    • help/help.89
    • help/help.9
    • help/help.90
    • help/help.91
    • help/help.92
    • help/help.94
    • help/help.95
    • help/help.96
    • help/help.97
    • help/help.98
    • help/help.99
    • help/helpfile
    • help/helpfiles
    • help/policy
    • help/spell.0
    • help/spell.1
    • help/spell.10
    • help/spell.11
    • help/spell.12
    • help/spell.13
    • help/spell.14
    • help/spell.15
    • help/spell.16
    • help/spell.17
    • help/spell.18
    • help/spell.19
    • help/spell.2
    • help/spell.21
    • help/spell.22
    • help/spell.23
    • help/spell.24
    • help/spell.25
    • help/spell.26
    • help/spell.27
    • help/spell.28
    • help/spell.29
    • help/spell.3
    • help/spell.30
    • help/spell.31
    • help/spell.32
    • help/spell.33
    • help/spell.34
    • help/spell.35
    • help/spell.36
    • help/spell.37
    • help/spell.38
    • help/spell.39
    • help/spell.4
    • help/spell.40
    • help/spell.41
    • help/spell.42
    • help/spell.43
    • help/spell.44
    • help/spell.45
    • help/spell.46
    • help/spell.48
    • help/spell.49
    • help/spell.5
    • help/spell.50
    • help/spell.51
    • help/spell.52
    • help/spell.53
    • help/spell.55
    • help/spell.6
    • help/spell.7
    • help/spell.8
    • help/spell.9
    • help/spellfile
    • help/welcome
  • b3ff282
    • README.md
    • docs/revive/baseline.md
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

이 PR은 프로젝트 전반에 걸쳐 CP949 인코딩 의존성을 제거하고 UTF-8을 기본으로 사용하도록 전환하는 중요한 리팩토링입니다. 문서 파일들의 인코딩을 변환하고, 새로운 UTF-8 정책을 README.md 등에 명확히 문서화한 점이 인상적입니다.
docs/** 범위 내의 레거시 문서에서 자동화된 인코딩 변환 과정에서 발생한 깨진 문자나 오타 등의 불일치가 발견되었으나, 프로젝트 정책에 따라 이러한 원본 레거시 문서는 현재 상태 그대로 유지하며 수정하지 않습니다. 따라서 해당 문서들에 대한 수정 제안은 제외되었습니다.

@comfuture

Copy link
Copy Markdown
Owner Author

Addressed the two open review threads in commit 2e75eb4.

  • Enforced UTF-8 player-name validation with both codepoint and byte limits (max 12 codepoints, max 14 bytes) to remain compatible with legacy fixed buffers.
  • Added defensive bounded copy during login name reload path.
  • Fixed UTF-8 prefix stripping for 모든... by using correct byte length, and applied the same fix to matching paths in bank.c.

Validation:

  • ./scripts/run-smoke.sh passed.
  • Manual check: 5-Hangul (15-byte) name is rejected with a clear message and prompt recovery.

@comfuture
comfuture merged commit d220df6 into master Feb 11, 2026
5 checks passed
@comfuture
comfuture deleted the feat/utf-8 branch February 11, 2026 10:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant