| 1 | <!-- Generated: 2026-03-27 | Updated: 2026-05-08 --> |
| 2 | |
| 3 | # douyin-downloader |
| 4 | |
| 5 | ## Purpose |
| 6 | A Python-based Douyin (TikTok China) batch downloader that fetches videos, galleries, music, and user content without watermarks. Supports multiple download modes (user posts, likes, mixes, music), concurrent downloads with rate limiting, cookie-based authentication, and optional Whisper transcription. CLI-driven with YAML configuration. |
| 7 | |
| 8 | ## Key Files |
| 9 | |
| 10 | | File | Description | |
| 11 | |------|-------------| |
| 12 | | `run.py` | Entry point — bootstraps `sys.path` and delegates to `cli.main:main()` | |
| 13 | | `__init__.py` | Package version (`2.0.0`) | |
| 14 | | `pyproject.toml` | Build config, dependencies, CLI entry point (`douyin-dl`), tool settings | |
| 15 | | `config.example.yml` | Example YAML config for users to copy and customize | |
| 16 | | `requirements.txt` | Pinned dependency list (mirrors pyproject.toml) | |
| 17 | | `Dockerfile` | Container build for the downloader | |
| 18 | | `PROJECT_SUMMARY.md` | Architecture overview document | |
| 19 | |
| 20 | ## Subdirectories |
| 21 | |
| 22 | | Directory | Purpose | |
| 23 | |-----------|---------| |
| 24 | | `auth/` | Cookie and MS token management (see `auth/AGENTS.md`) | |
| 25 | | `cli/` | CLI argument parsing, main async loop, progress display (see `cli/AGENTS.md`) | |
| 26 | | `config/` | YAML config loading, env var overrides, defaults (see `config/AGENTS.md`) | |
| 27 | | `control/` | Concurrency control — rate limiter, retry handler, queue manager (see `control/AGENTS.md`) | |
| 28 | | `core/` | Business logic — API client, URL parser, downloaders, strategy pattern (see `core/AGENTS.md`) | |
| 29 | | `storage/` | SQLite database, file management, metadata handling (see `storage/AGENTS.md`) | |
| 30 | | `tests/` | Pytest test suite with 23 test modules (see `tests/AGENTS.md`) | |
| 31 | | `tools/` | Standalone utilities like browser-based cookie fetching (see `tools/AGENTS.md`) | |
| 32 | | `utils/` | Shared helpers — logging, validation, anti-bot signatures (see `utils/AGENTS.md`) | |
| 33 | |
| 34 | ## For AI Agents |
| 35 | |
| 36 | ### Working In This Directory |
| 37 | - Python 3.8+ compatibility required — avoid walrus operator, `match` statements, and `type` aliases |
| 38 | - All I/O is async (`aiohttp`, `aiofiles`, `aiosqlite`) — never use blocking I/O in core paths |
| 39 | - Entry point is `cli.main:main()` which calls `asyncio.run(main_async(args))` |
| 40 | - Config is YAML-based with env var overrides (`DOUYIN_*` prefix) |
| 41 | - The `mix`/`allmix` config alias system requires special handling (see `config/config_loader.py`) |
| 42 | |
| 43 | ### Shared Logic With Desktop |
| 44 | - This project shares Python backend logic with `/Users/crimson/codes/douyin/douyin-downloader-desktop`. |
| 45 | - When fixing shared logic in `auth/`, `cli/`, `config/`, `control/`, `core/`, `storage/`, `tools/`, `utils/`, or shared tests, apply the equivalent fix in both projects unless the difference is explicitly desktop-only or CLI-only. |
| 46 | - Before finishing a shared-logic fix, compare the touched shared files against the sibling project and either keep them identical or document the intentional divergence. |
| 47 | - **Sync script:** `../douyin-downloader-desktop/scripts/sync-to-cli.sh` copies all shared files from the desktop project here. Run `--check` to detect drift. |
| 48 | - **Intentional divergences** (these files differ by design): |
| 49 | - `cli/main.py` — CLI omits desktop-only `_verify_self_checksum()` and `_enforce_license_at_startup()`. |
| 50 | - `run.py` — CLI is a simple bootstrap; desktop has sidecar startup + data-dir migration. |
| 51 | - `server/app.py`, `server/jobs.py` — CLI server is a simplified subset; desktop adds license, SSE, overrides, cancel. |
| 52 | - `control/__init__.py` — CLI doesn't export `ProgressReporter` classes (desktop UI only). |
| 53 | |
| 54 | ### Testing Requirements |
| 55 | - Run: `python -m pytest tests/` |
| 56 | - Async tests use `pytest-asyncio` with `asyncio_mode = "auto"` |
| 57 | - Linting: `ruff check .` (target Python 3.8, line-length 100) |
| 58 | |
| 59 | ### Common Patterns |
| 60 | - Factory pattern for downloaders (`DownloaderFactory.create()`) |
| 61 | - Strategy pattern for user download modes (`core/user_modes/`) |
| 62 | - Registry pattern for mode discovery (`UserModeRegistry`) |
| 63 | - All downloaders inherit from `BaseDownloader` with shared `_download_mode_items()` |
| 64 | - Logging via `utils.logger.setup_logger(name)` — one logger per module |
| 65 | |
| 66 | ## Dependencies |
| 67 | |
| 68 | ### External |
| 69 | - `aiohttp` — async HTTP client for API calls and downloads |
| 70 | - `aiofiles` — async file I/O |
| 71 | - `aiosqlite` — async SQLite for download history |
| 72 | - `rich` — terminal UI (progress bars, tables, styled output) |
| 73 | - `pyyaml` — YAML config parsing |
| 74 | - `python-dateutil` — date/time parsing for time-range filters |
| 75 | - `gmssl` — Chinese SM3/SM4 crypto for anti-bot signatures |
| 76 | |
| 77 | ### Optional |
| 78 | - `playwright` — browser automation for cookie fetching |
| 79 | - `openai-whisper` — audio transcription |
| 80 | |
| 81 | <!-- MANUAL: --> |
| 82 |