Skip to content

Feature/bili article support#932

Open
ChenJiahao1 wants to merge 12 commits into
NanmiCoder:mainfrom
ChenJiahao1:feature/bili-article-support
Open

Feature/bili article support#932
ChenJiahao1 wants to merge 12 commits into
NanmiCoder:mainfrom
ChenJiahao1:feature/bili-article-support

Conversation

@ChenJiahao1

Copy link
Copy Markdown

背景

Fixes #853

本次 PR 针对哔哩哔哩专栏链接在 detail 模式下无法正常抓取的问题,补齐 B 站专栏从 ID/URL 解析、详情抓取、评论抓取到多存储后端落库的完整链路。

主要改动

  • 支持在 BILI_SPECIFIED_ID_LIST / --specified_id 中传入 B 站专栏:
    • https://www.bilibili.com/read/cv123456
    • cv123456
    • 123456
  • 支持视频和专栏 ID 混合传入,detail 模式会自动拆分并分别走视频/专栏抓取流程。
  • 新增 B 站专栏详情抓取:
    • 调用 /x/article/view
    • 保存专栏标题、摘要、正文、作者信息、点赞/收藏/分享/评论数等字段。
  • 泛化 B 站评论抓取逻辑:
    • 视频评论继续使用 type=1
    • 专栏评论新增使用 type=12
    • 复用一级评论、二级评论、分页、数量限制和回调存储流程。
  • 新增专栏数据存储支持:
    • DB:新增 bilibili_articlebilibili_article_comment 模型
    • CSV / JSON / JSONL / MongoDB:新增专栏正文和专栏评论写入入口
  • 更新配置和 CLI 帮助文案,明确 specified_id 支持 article ID。
  • 补充测试覆盖:
    • 专栏 URL / cv ID / 纯数字 ID 解析
    • 混合视频/专栏 ID 分流
    • 专栏详情 API 调用
    • 视频/专栏评论 type 参数
    • 专栏抓取流程与评论回调
    • 专栏 DB 模型和存储字段映射

测试

.\.venv\Scripts\python.exe -m pytest tests/database/test_bilibili_article_models.py tests/media_platform/bilibili/test_client_article.py tests/media_platform/bilibili/test_client_comments.py tests/media_platform/bilibili/test_core_article_flow.py tests/media_platform/bilibili/test_help.py tests/store/bilibili/test_article_store.py tests/test_bilibili_client_comments.py tests/test_cmd_arg_bilibili.py

@ChenJiahao1 ChenJiahao1 requested a review from NanmiCoder as a code owner July 2, 2026 06:24
@dosubot dosubot Bot added size:XL This PR changes 500-999 lines, ignoring generated files. enhancement New feature or request labels Jul 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request size:XL This PR changes 500-999 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Bilibili专栏评论区无法获取

1 participant