| 1 | # last30days Skill |
| 2 | |
| 3 | Agent Skills package for researching any topic across Reddit, X, YouTube, and web. Installable across Claude Code (most common host), Codex, Cursor, GitHub Copilot, Gemini CLI, Grok (xAI), and 50+ other [Agent Skills](https://agentskills.io) hosts. Python scripts with multi-source search aggregation. |
| 4 | |
| 5 | ## Structure |
| 6 | - `skills/last30days/SKILL.md` — canonical skill definition / runtime spec the model reads when the slash command fires |
| 7 | - `skills/last30days/scripts/last30days.py` — main research engine |
| 8 | - `skills/last30days/scripts/lib/` — search, enrichment, rendering modules |
| 9 | - `skills/last30days/scripts/lib/vendor/bird-search/` — vendored X search client |
| 10 | - `docs/solutions/` — documented solutions to past problems (bugs, best practices, workflow patterns), organized by category with YAML frontmatter (`module`, `tags`, `problem_type`) |
| 11 | - `CONCEPTS.md` — shared domain vocabulary (Skill, Engine, Harness, Beta channel) — relevant when orienting to the codebase or discussing project terminology |
| 12 | - `CONFIGURATION.md` — user-facing knobs (env vars, flags, per-host install patterns); keep in sync per the rules below |
| 13 | - `CHANGELOG.md` — structured release history built by towncrier at release time (launch copy lives in GitHub Releases) |
| 14 | - `changelog.d/` — per-PR news fragments; feature PRs write here, never edit `CHANGELOG.md` directly |
| 15 | - `CONTRIBUTING.md` — setup, fragments, and release notes for humans and agents (towncrier is release-only) |
| 16 | - `.github/scripts/prepare_release.py` — lockstep version bump + towncrier build (release PRs only) |
| 17 | - `HERMES_SETUP.md` — install instructions for the Hermes harness specifically |
| 18 | |
| 19 | ## Orientation |
| 20 | - This is an Agent Skills package, not a CLI tool. The product is the slash-command-invoked skill (`/last30days <topic>` in most harnesses); `scripts/last30days.py` is implementation. Claude Code is the most common host but not the only one — features must work across every harness the skill installs into. |
| 21 | - Feature design starts from the slash-command UX. A new engine flag with no SKILL.md integration is incomplete — the model invoking the skill won't know the flag exists. |
| 22 | - README and PR examples show `/last30days <topic>` first. Direct CLI invocation (`python3 scripts/last30days.py ...`) is a fallback for scripting, cron, and dev-time engine testing; label it as such, never as the primary path. |
| 23 | - Slash commands don't pass shell mechanics through. `/last30days OpenClaw --emit=html | pbcopy` is invalid in any harness — either use the slash form (no flags or pipes; let the model translate user intent into engine flags) or use the direct CLI form (full `python3 ...` with explicit flags and a real shell). |
| 24 | |
| 25 | ## Commands |
| 26 | ```bash |
| 27 | # Dev/fallback: direct engine invocation (scripting, cron, or engine testing only). |
| 28 | # Saves to $LAST30DAYS_MEMORY_DIR when set in shell or ~/.config/last30days/.env; |
| 29 | # add --save-dir <path> for a one-off override. Mirrors LAST30DAYS_STORE convention. |
| 30 | python3 skills/last30days/scripts/last30days.py "test query" --emit=compact |
| 31 | npx skills add . -g -y # copies skill into ~/.agents/skills/<name>/ (frozen at install time); re-run to sync working-tree edits — see Rules below |
| 32 | |
| 33 | # Tests (pytest, ~89 files under tests/, configured in pyproject.toml) |
| 34 | uv run pytest # full suite |
| 35 | uv run pytest tests/test_dedupe_v3.py # single file |
| 36 | uv run pytest tests/test_dedupe_v3.py -k some_case # single case |
| 37 | uv run pytest --cov # with coverage (skips lib/vendor/) |
| 38 | |
| 39 | # Release prep (maintainers / release automation — not feature PRs): |
| 40 | # Prefer GitHub Actions → "Prepare release". Local equivalent: |
| 41 | uv run python .github/scripts/prepare_release.py --bump patch # or --version X.Y.Z |
| 42 | ``` |
| 43 | |
| 44 | Python 3.12+ required. Use `uv` for the env; the venv lives at `.venv/`. |
| 45 | |
| 46 | ## Changelog and releases (agents) |
| 47 | |
| 48 | Agents open most PRs. Follow this so `CHANGELOG.md` stops conflicting and versions stay lockstep: |
| 49 | |
| 50 | 1. **Feature/fix PRs:** add `changelog.d/<pr-or-issue>.<type>.md` (`added` / `changed` / `fixed` / `removed` / `deprecated` / `security`) when the change belongs in the next release notes. See `changelog.d/README.md` and `CONTRIBUTING.md`. Fill the PR template’s Summary, Agent disclosure, and Relationship sections. |
| 51 | 2. **Never** edit `CHANGELOG.md` in a feature PR. **Never** bump version strings in `pyproject.toml`, `SKILL.md`, plugin/marketplace JSON, or `uv.lock` outside a release PR. CI (`changelog-guard.yml`) enforces this. |
| 52 | 3. **Nothing for release notes:** omit the fragment, check Skip changelog in the template, and add the `skip-changelog` label. |
| 53 | 4. **Cutting a release:** run Actions → **Prepare release** (patch/minor/major). That opens a `chore(release): bump version to X.Y.Z` PR which runs towncrier and bumps every lockstep surface. Merging to `main` triggers **Tag release**, which pushes `vX.Y.Z` and existing `release.yml` publishes `.skill` / `.mcpb` artifacts. Do not hand-edit ten version files. Contributors do not need a global towncrier install — `uv sync --group dev` (or the Action) provides it for release prep only. |
| 54 | 5. Lockstep gate remains `tests/test_plugin_contract.py::test_versions_match_across_manifests`. Workflow contract: `tests/test_changelog_workflow.py`. |
| 55 | |
| 56 | ## Rules |
| 57 | - `lib/__init__.py` must be bare package marker (comment only, NO eager imports) |
| 58 | - One-time setup: `npx skills add . -g -y` copies the skill into `~/.agents/skills/<name>/` (real directory) and, for harnesses that support symlinked skill dirs, drops a per-host symlink pointing at that copy. **Working-tree edits do NOT propagate automatically** — the `~/.agents/skills/<name>/` copy is frozen at install time. To sync after edits, re-run `npx skills add . -g -y`. For live-edit on a dev machine, replace the install copy with a symlink to the working tree: `ln -sfn "$PWD/skills/last30days" ~/.agents/skills/last30days` (run from the repo root). |
| 59 | - Git remote: origin = public (`mvanhorn/last30days-skill`) |
| 60 | - Do not reduce `fail_under` in `pyproject.toml` (`[tool.coverage.report]`) without documenting why in the PR. The coverage gate is a floor meant to rise over time, not to be relaxed when new code is under-tested. |
| 61 | - Every `lib/*.py` call to `log.source_log(...)` must pass `tty_only=False`. The default is `True`, which silently drops every line when stderr isn't a TTY (Claude Code, Codex, CI, captured output) — turning source observability into invisible failure. Enforced by `tests/test_source_log_visibility.py`. |
| 62 | - **CLI-gated optional sources** (Digg via `digg-pp-cli`, YouTube via `yt-dlp`) activate only when `shutil.which` resolves the binary on the **agent subprocess PATH** — not merely when the file exists on disk. First-run setup installs Digg through `@mvanhorn/printing-press-library` (default `$HOME/.local/bin`); Hermes/OpenClaw gateways often need that directory on PATH. Setup must distinguish PATH-visible installs from off-PATH binaries and must not claim "now active" unless the engine gate would pass. See `docs/solutions/integration-issues/digg-cli-agent-path-setup-wizard.md`. |
| 63 | - **First-run onboarding is consent-driven, model-led, and host-split.** The setup subprocess does only mechanical work (cookie reads, tool installs, GitHub device-auth, and emitting the engine-owned welcome via `--welcome`) — it cannot prompt, so consent lives in `SKILL.md` Step 0. Two flows avoid model-authored prose that Claude Code folds or the model skips: in the **Modal Flow** the welcome pitch is embedded in the setup modal's question (the AskUserQuestion modal is the only always-fully-visible surface — a separate welcome message or `--welcome` Bash run gets buried behind "ctrl+o to expand"); the **Non-Modal Prose Flow** still uses `last30days.py --welcome` (relayed verbatim) since it has no modal. The GitHub device code is surfaced by a two-command split — `setup --github-start` returns the code fast (foreground, copies to clipboard) and `setup --github-poll` waits for authorization (`setup --github` still chains both for back-compat). Step 0 has TWO branches: a **Claude Code Modal Flow** (the restored v3.0.0 `AskUserQuestion`-driven NUX — welcome, Auto/Manual/Skip, cookie consent, ScrapeCreators offer, `INCLUDE_SOURCES` opt-in, first-topic picker) for hosts with modals, and a **Non-Modal Prose Flow** for hosts without (OpenClaw, Codex, Cursor, Gemini CLI, Grok). Both ask before reading cookies, surface the macOS Full Disk Access fix on permission-denied, and offer the ScrapeCreators GitHub signup (10,000 free calls) on every first run. A successful `setup --github` persists `SCRAPECREATORS_API_KEY` automatically (via `setup_wizard.write_api_key`, 0o600) and masks the key in stdout. Do NOT collapse the modal flow back into a bare silent `setup` call or flatten it to prose-only — the guided modals are the feature (they eroded once and were restored). The onboarding contract is locked by `tests/test_onboarding_contract.py`. The Step 5 source opt-in is two tiers, both comment-enabled: **Recommended** (TikTok + Instagram posts AND top comments, plus YouTube comments — `INCLUDE_SOURCES=tiktok,instagram,youtube_comments,tiktok_comments,instagram_comments`) and **Everything** (also Threads + Pinterest). Comments are on by default (posts on → comments on for all three platforms); **Threads and Pinterest are the only opt-in extras**, appearing only in the Step 5 Everything option, never in the welcome or the Step 4 offer. Instagram comments are fetched via ScrapeCreators (`/v2/instagram/post/comments`, ranked by `comment_like_count`) with full vote-weighting parity to YouTube/TikTok (a dedicated `_instagram_engagement` carve-out, the `_VOTE_LOG_REFERENCE`/label/threshold entries). The cross-platform "Top Community Comments" list (`render._render_top_comments`) selects **round-robin by within-platform rank** (every platform's #1, then #2, then #3) so a viral platform can't crowd out a smaller one, and drops the per-platform absolute floor so a less-watched video's killer low-vote comment still surfaces. |
| 64 | |
| 65 | ## Security hygiene |
| 66 | - Never commit real API keys, browser cookies, auth tokens, app passwords, access tokens, or `.env` contents. |
| 67 | - Use the env-based auth patterns in `skills/last30days/scripts/lib/env.py`; tests and fixtures must use obvious dummy values only. |
| 68 | - Keep examples safe by redacting secrets and avoiding copy/pasteable live credentials in docs, fixtures, and test data. |
| 69 | - Do not weaken or disable the advisory security workflow (`.github/workflows/security.yml`) without explaining why in the PR description or review thread. |
| 70 | |
| 71 | ## Maintaining README translations |
| 72 | |
| 73 | `README.md` is the canonical English README. When changing it, reflect the same substantive updates in `README.fr.md`, `README.de.md`, `README.es.md`, `README.pt-BR.md`, `README.ja.md`, and `README.zh-CN.md`, preserving commands, links, tables, and reciprocal language navigation. |
| 74 | |
| 75 | ## Maintaining CONFIGURATION.md |
| 76 | |
| 77 | `CONFIGURATION.md` is the user-facing configuration reference — save paths, per-source API keys, web-search backend priority, trend-monitoring stack, per-client install patterns. Distinct from `SKILL.md` (the canonical runtime spec). |
| 78 | |
| 79 | Update `CONFIGURATION.md` when: |
| 80 | |
| 81 | - adding a new env var (e.g. `LAST30DAYS_*`, `BSKY_*`, `*_API_KEY`) |
| 82 | - adding a new CLI flag that affects configuration (e.g. `--store`, `--web-backend`) |
| 83 | - adding a new per-client install pattern (Claude Code, Gemini, Codex, Cursor, Grok, Hermes…) |
| 84 | - adding a new optional source that requires its own credential |
| 85 | - changing the priority order of config layers (per-run flag > env > `.env` file > defaults) |
| 86 | |
| 87 | Keep the existing structure organized by how often each layer is touched: per-run flags → env vars / `.env` → optional trend-monitoring stack → per-client patterns. Add new content into the right section rather than appending at the end. |
| 88 | |
| 89 | When a new config concept lands in `SKILL.md` or `AGENTS.md`, mirror the user-facing knob in `CONFIGURATION.md` so non-agent readers can configure the skill without reverse-engineering it from the runtime spec. |
| 90 | |
| 91 | ## Plugin manifests (Grok) |
| 92 | |
| 93 | The repo doubles as a native Grok Build plugin via `.grok-plugin/plugin.json` + `.grok-plugin/marketplace.json`. Grok also reads `.claude-plugin/*` for compatibility; the native pair is the first-class lane and what an official xAI marketplace listing points at. The self-hosted catalog uses a bare Git URL source (`{"source":"url","url":"https://github.com/mvanhorn/last30days-skill.git"}`) so `grok plugin marketplace add mvanhorn/last30days-skill` tracks HEAD — not a self-referential local `path: "."` (Grok does not enumerate those). Version lockstep with Claude/Codex/Gemini manifests is enforced by `tests/test_plugin_contract.py`. Validate with `grok plugin validate .`. |
| 94 | |
| 95 | ## Submitting to the xAI plugin marketplace |
| 96 | |
| 97 | Getting last30days into xAI's official catalog (`xai-org/plugin-marketplace`) is an outbound PR to *their* repo — an index that only points at our source, so nothing of last30days is vendored there. Do this **after** the change you want to ship has merged to `main`: the entry pins a commit that must already exist. |
| 98 | |
| 99 | 1. Fork `xai-org/plugin-marketplace` and branch from `main`. |
| 100 | 2. Get the commit to pin — a full 40-char lowercase SHA; a branch, tag, or short SHA is rejected by their validator: |
| 101 | ```bash |
| 102 | git ls-remote https://github.com/mvanhorn/last30days-skill.git HEAD |
| 103 | ``` |
| 104 | 3. Add one entry to their `.grok-plugin/marketplace.json` under `plugins[]`, a remote source pinned to that SHA: |
| 105 | ```json |
| 106 | { |
| 107 | "name": "last30days", |
| 108 | "description": "Research any topic across Reddit, X, YouTube, TikTok, Instagram, Hacker News, Polymarket, GitHub, and 5+ more sources. AI agent scores by upvotes, likes, and real money - not editors.", |
| 109 | "category": "productivity", |
| 110 | "source": { |
| 111 | "source": "url", |
| 112 | "url": "https://github.com/mvanhorn/last30days-skill.git", |
| 113 | "sha": "<full-40-char-sha-from-step-2>" |
| 114 | }, |
| 115 | "homepage": "https://github.com/mvanhorn/last30days-skill", |
| 116 | "keywords": ["last30days", "last 30 days"] |
| 117 | } |
| 118 | ``` |
| 119 | 4. Regenerate their component index (never hand-edit it) and validate exactly as their CI does: |
| 120 | ```bash |
| 121 | python3 scripts/generate-plugin-index.py |
| 122 | python3 scripts/validate-catalog.py |
| 123 | python3 scripts/generate-plugin-index.py --check |
| 124 | ``` |
| 125 | 5. Open the PR, fill in their template, and wait for code-owner review. |
| 126 | |
| 127 | To roll out a later update in their catalog, bump the pinned `sha` in the existing entry — never open a second, parallel entry. |
| 128 | |
| 129 | Do not confuse this with our own `.grok-plugin/marketplace.json`: that file makes this repo directly addable as a Grok marketplace (`grok plugin marketplace add mvanhorn/last30days-skill`) and uses a **bare URL** source (no SHA) so it tracks HEAD; the xAI entry above lives in *their* repo and uses a **remote** source pinned to a SHA. |
| 130 | |
| 131 | ## Beta channel |
| 132 | |
| 133 | Experimental changes get tested on `mvanhorn/last30days-skill-private`, which installs as a parallel `/last30days-beta` slash command. Beta-only changes never ship to public without a review PR here. Workflow guide lives at `BETA.md` in the private repo. Plan that established this setup: `docs/plans/2026-04-17-005-feat-beta-skill-from-private-repo-plan.md`. |
| 134 |