Skip to content

Remove spreadsheet caching, externalize AUTH_KEY, and enhanced batch processing#1400

Open
matikhi wants to merge 8 commits into
therealaleph:mainfrom
matikhi:main
Open

Remove spreadsheet caching, externalize AUTH_KEY, and enhanced batch processing#1400
matikhi wants to merge 8 commits into
therealaleph:mainfrom
matikhi:main

Conversation

@matikhi
Copy link
Copy Markdown

@matikhi matikhi commented May 26, 2026

توضیحات

این PR شامل بهبودهای عمده‌ای در کارایی، امنیت و پایداری کد است:

تغییرات اصلی

  1. حذف سیستم کش اسپریدشیت — سیستم کش مبتنی بر اسپریدشیت حذف شد تا مصرف quota و latency کاهش یابد
  2. امنیت AUTH_KEY — کلید احراز هویت از hardcoded کد خارج شد و از PropertiesService استفاده شد
  3. پشتیبانی Exit Node — مشکل نمایش raw JSON در مرورگرها برای پاسخ‌های exit_node حل شد
  4. بهبود کارایی — از Set به جای Array استفاده شد برای lookups سریع‌تر
  5. پردازش بسته‌ای — requests به بسته‌های 50 آیتمی تقسیم شد تا از محدودیت‌های GAS fetchAll جلوگیری شود

تغییرات Breaking

  1. مهم: AUTH_KEY باید در Script Properties تنظیم شود
  2. برای عبور از چلنج Cloudflare، آدرس "challenges.cloudflare.com" باید به hosts config اضافه شود

جزئیات فنی

  • تعویض Array با Map برای مدیریت batch processing بهتر
  • Refactor _doBatch برای پشتیبانی از batch processing
  • اضافه شدن safety checks در چندین مکان
  • تغییر نام توابع برای وضوح بیشتر

matikhi added 8 commits May 26, 2026 10:19
Remove the spreadsheet caching layer that was creating performance bottlenecks and quota consumption. This simplifies the codebase and eliminates unnecessary I/O latency from cache reads/writes.

Issues addressed:
- Excessive quota consumption from spreadsheet operations
- Added latency from cache lookups and writes
- Complexity in cache invalidation logic

BREAKING CHANGE: The optional caching mechanism has been removed. Applications relying on the caching parameter should implement their own caching layer or discard cacheing.
Move AUTH_KEY from hardcoded value to PropertiesService.getScriptProperties() to prevent accidental credential exposure in source control and reduce security risk.

- Store sensitive AUTH_KEY in script properties instead of codebase
- Updated implementation comments to guide users on setup
- Eliminates risk of unintended credential sharing via version control

BREAKING CHANGE: AUTH_KEY must now be configured via script properties. Users have to:
1. Open Project Settings > Script Properties
2. Add key-value pair: AUTH_KEY = <your_key_value>
…challenges

Fix issue where exit_node was rendering raw JSON responses in browsers instead of properly handling responses.
Replace all var declarations with let for consistency.
Fix wrong property of ContentService.MimeType (HTML->XML)

- In order to pass Cloudflare anti-bot challenges ,Users have to add "challenges.cloudflare.com" to hosts configuration in client app
  - DenoDeploy passes Cloudflare tests successfully
…zed logic

- Replace Array with Set for O(1) lookups
- Optimize single-item fetch to avoid fetchAll
- Rename function and arguments for clarity
- Extract duplicated logic into helper function
- Add safety checks throughout codebase
Split requests into batches of 50 items to prevent hitting Google Apps Script fetchAll method limitations.

- Add batching logic using Map structure to avoid lists index syncronization overhead
- Refactor _doBatch core logic for batch processing
- Add helper function for batch handling
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant