| 1 | --- |
| 2 | description: Generate source-intake stage that fills factual gaps and records adopted webpage provenance before planning or direct SVG authoring. |
| 3 | --- |
| 4 | |
| 5 | # Topic Research Stage |
| 6 | |
| 7 | > Factual preparation inside the active Generate profile's source intake. |
| 8 | > Default Generate hands its output to Strategist; Quick Generate's main agent |
| 9 | > consumes the same output. Run immediately for topic-only input, or after |
| 10 | > supplied material is converted and read when it leaves planning-critical |
| 11 | > factual gaps. Output is exactly a research supplement plus stable fact |
| 12 | > provenance for project import. Adopted webpage URLs remain in the provenance |
| 13 | > file and are not expanded during the project-initialization handoff. |
| 14 | |
| 15 | This stage supplies facts needed to build the requested deck and records the |
| 16 | webpages actually adopted during that research. It makes no deck image |
| 17 | selection and performs no independent image search or generation. The facts |
| 18 | JSON is provenance, not a page-download queue: `project_manager.py |
| 19 | import-sources` imports the research pair without fetching its `source_url` |
| 20 | values. A page may be fetched later only for the bounded image fallback below. |
| 21 | |
| 22 | ## When to Run |
| 23 | |
| 24 | | Material state | Action | |
| 25 | |---|---| |
| 26 | | Topic or requirements with no supporting facts | Research the factual baseline needed for the requested outcome | |
| 27 | | Supplied files or chat content cover only part of the requested outcome | After conversion and reading, research only the identified externally verifiable gaps | |
| 28 | | Supplied material already supports the requested outcome | Skip this stage and continue the active Generate profile's source preparation | |
| 29 | | User requires a closed corpus, source-only transformation, or no external enrichment | Skip this stage and keep planning within supplied material | |
| 30 | |
| 31 | **Sufficiency test**: a gap exists when the active content owner would otherwise need to invent, omit, or leave unsupported an externally verifiable claim required by the user's requested outcome. File presence, source length, and a generic topic taxonomy do not decide sufficiency. |
| 32 | |
| 33 | **Hard rule — preserve supplied facts**: supplement the user's material; never |
| 34 | silently replace it. Record a material source conflict in the research output |
| 35 | for the active content owner instead of choosing a different claim without |
| 36 | disclosure. Do not research omissions outside the requested scope. |
| 37 | |
| 38 | --- |
| 39 | |
| 40 | ## Step 1: Define the gap brief |
| 41 | |
| 42 | **Clarification boundary**: Default Generate bundles only genuinely missing |
| 43 | scope or research-boundary decisions into one clarifier. Quick Generate applies |
| 44 | the defaults below and continues without interaction; stop only when a required |
| 45 | permission or safety boundary cannot be inferred responsibly. Skip clarification |
| 46 | when the request and supplied material are already clear. |
| 47 | |
| 48 | | Item | Default if unspecified | |
| 49 | |---|---| |
| 50 | | Topic | From the user request | |
| 51 | | Requested scope / outcome | From the user request; otherwise broad overview | |
| 52 | | Supplied-material baseline | Facts and claims already available | |
| 53 | | Research gaps | Only facts needed to support the requested outcome | |
| 54 | | External-source boundary | External factual enrichment allowed; supplied facts remain authoritative inputs | |
| 55 | | Output language | Match user input | |
| 56 | | Target audience / communication intent | Use what is explicit; Default leaves final confirmation to Strategist, while Quick resolves routine gaps in active context | |
| 57 | | Research stem (`<research_slug>`) | `<topic_slug>_research`; choose another unused snake_case stem rather than overwrite an existing file | |
| 58 | |
| 59 | Do not repeat the full default-pipeline confirmation here. Default Generate |
| 60 | confirms the complete communication contract in Step 4; Quick Generate adds no |
| 61 | confirmation stage. |
| 62 | |
| 63 | --- |
| 64 | |
| 65 | ## Execution Context |
| 66 | |
| 67 | **Default — isolated research when available**: The main agent owns the sufficiency decision and gap brief. When the current AI editor supports and permits an isolated subagent with web/fetch access and write access to the declared outputs, dispatch exactly one research worker. Otherwise the main agent runs Steps 2–3 locally. |
| 68 | |
| 69 | | Actor | Contract | |
| 70 | |---|---| |
| 71 | | Main agent | Supply the topic/outcome, baseline or relevant source paths, declared gaps, output language, two exact unused output paths, and this stage's absolute path as execution authority; use paths instead of pasting source bodies when possible | |
| 72 | | Research worker | Read the supplied stage file completely, then follow Steps 2–3 using the brief and declared source paths as its baseline; limit project writes to the two output artifacts; perform no independent image search/generation and make no deck-planning, image-selection, or design decisions | |
| 73 | |
| 74 | **Hard rule — isolate retrieval, not research**: Raw page content and fetch transcripts stay in the worker context. The 250-word limit applies only to its chat receipt: return `status`, exact artifact paths, covered/unresolved gap counts, external-fact count, and material conflicts. It does not cap or replace the two artifacts. After validation and import, the active content owner reads the complete imported research supplement and fact-provenance JSON into the main context before planning or direct SVG authoring; never use the receipt or validation summary as content. |
| 75 | |
| 76 | **Validation**: Before import, the main agent verifies both exact files exist, the Markdown contains `## Research Brief` and no source list or URL, the JSON parses with schema `ppt-master.fact-provenance.v1` and unique sequential IDs, and the two files agree. Return an invalid pair to the research worker for owning-artifact repair; use main-context web research only when isolated execution is unavailable. |
| 77 | |
| 78 | --- |
| 79 | |
| 80 | ## Step 2: Gather factual sources |
| 81 | |
| 82 | Use the web search and fetch tools available in the active research context. An isolated worker without them returns `blocked: web-tools-unavailable`. If no usable research context has search/fetch tools, the main agent pauses and asks the user for authoritative URLs covering the declared gaps, then fetches each with: |
| 83 | |
| 84 | ```bash |
| 85 | python3 ${SKILL_DIR}/scripts/source_to_md/web_to_md.py <URL> \ |
| 86 | -o projects/<research_slug>_web_sources/<source_slug>.md --no-images |
| 87 | ``` |
| 88 | |
| 89 | Preserve the resulting Markdown and conversion profile for research. Remote |
| 90 | inline-image links remain in the Markdown; no image files are downloaded. |
| 91 | |
| 92 | | Phase | Action | |
| 93 | |---|---| |
| 94 | | Orient | Search only far enough to map authoritative sources to the declared gaps | |
| 95 | | Deep fetch | Read the highest-signal primary or authoritative pages in full | |
| 96 | | Targeted fill | Search only for gaps still unsupported after those reads | |
| 97 | |
| 98 | | Priority | Source | |
| 99 | |---|---| |
| 100 | | 1 | Primary sources, official sites, institutional releases, standards, or original research | |
| 101 | | 2 | Authoritative reference works and reputable academic sources | |
| 102 | | 3 | Reputable reporting or analysis when primary evidence is unavailable | |
| 103 | | Avoid | Unsourced reposts, unverifiable summaries, and stock-aggregator pages | |
| 104 | |
| 105 | **Adopted webpage boundary**: Record a page URL only in the matching fact's |
| 106 | `source_url`, and only when it materially supports that retained fact. Do not |
| 107 | retain a page merely because its images may be useful, and do not add unopened |
| 108 | search results or pages found through a separate image-search pass. |
| 109 | |
| 110 | **Stop condition**: stop when every declared gap has enough sourced evidence for |
| 111 | the active content owner to decide whether and how to include it. Do not expand |
| 112 | into unrelated overview / history / outlook sections merely to make the |
| 113 | research look complete. |
| 114 | |
| 115 | --- |
| 116 | |
| 117 | ## Step 3: Save the factual supplement |
| 118 | |
| 119 | Write two artifacts under `projects/`: |
| 120 | |
| 121 | | Artifact | Path | |
| 122 | |---|---| |
| 123 | | Research supplement | `projects/<research_slug>.md` | |
| 124 | | Fact provenance | `projects/<research_slug>.facts.json` | |
| 125 | |
| 126 | **Hard rule — location and preservation**: write both files under `projects/`, never the repository root. Do not overwrite an existing user file; choose a new research stem instead. Do not create a research-image manifest or download embedded images. |
| 127 | |
| 128 | Begin the research Markdown with a compact `## Research Brief` containing the supplied-material baseline, declared gaps, audience / intent already known, and requested outcome. Organize the body by gap, include concrete facts only, flag material conflicts, and cite claims by `fact_id`. Do not add `## Sources` or URLs; the facts JSON is the only URL authority. |
| 129 | |
| 130 | Write every externally sourced claim that may enter the deck to `<research_slug>.facts.json` with a stable sequential ID, especially quantitative, date, ranking, attribution, and named-entity claims. Do not include user-supplied claims or invented scenario values. When no external claim is retained, write the schema with an empty `facts` array. |
| 131 | |
| 132 | ```json |
| 133 | { |
| 134 | "schema": "ppt-master.fact-provenance.v1", |
| 135 | "topic": "<topic>", |
| 136 | "facts": [ |
| 137 | { |
| 138 | "fact_id": "F001", |
| 139 | "claim": "One concise, presentation-ready factual claim", |
| 140 | "source_title": "Authoritative page title", |
| 141 | "source_url": "https://example.org/source", |
| 142 | "classification": "external", |
| 143 | "retrieved_at": "YYYY-MM-DD" |
| 144 | } |
| 145 | ] |
| 146 | } |
| 147 | ``` |
| 148 | |
| 149 | IDs are immutable within the file. Correct a claim under the same ID; never reuse a removed ID for a different fact. The research Markdown and provenance file must agree. |
| 150 | |
| 151 | --- |
| 152 | |
| 153 | ## Hand-off |
| 154 | |
| 155 | After project initialization, import the research pair and user-supplied |
| 156 | sources. The facts JSON is imported as an ordinary source file; its `source_url` |
| 157 | values are never expanded, so this command performs no webpage retrieval. |
| 158 | |
| 159 | ```bash |
| 160 | python3 ${SKILL_DIR}/scripts/project_manager.py import-sources \ |
| 161 | projects/<project_name> [<source_paths...>] \ |
| 162 | projects/<research_slug>.md projects/<research_slug>.facts.json |
| 163 | ``` |
| 164 | |
| 165 | If planning later exposes a required factual gap, return to this stage and |
| 166 | repair the research supplement plus facts JSON before continuing. Do not let |
| 167 | Strategist or Quick consume a newly fetched claim without updating that pair. |
| 168 | |
| 169 | Only after normal web-image providers, ranked thumbnail pages, and materially |
| 170 | different queries fail may an image owner with visual capability select one |
| 171 | relevant `source_url` from the facts JSON and fetch that one webpage package: |
| 172 | |
| 173 | ```bash |
| 174 | python3 ${SKILL_DIR}/scripts/source_to_md/web_to_md.py "<source_url>" \ |
| 175 | -o <project_path>/sources/<source_slug>.md |
| 176 | ``` |
| 177 | |
| 178 | This writes the page Markdown, conversion profile, and companion |
| 179 | `<source_slug>_files/` image package with `image_manifest.json`. Review that |
| 180 | package, then copy only accepted image files into `<project_path>/images/`; |
| 181 | leave every rejected or unused file in the source package. Fetch another page |
| 182 | only after the current package has no usable image. Do not pass the URL to |
| 183 | `project_manager.py import-sources`, which would promote every companion image |
| 184 | into the runtime pool. Without vision, skip this fallback and retain |
| 185 | `Needs-Manual`. |
| 186 | |
| 187 | The imported research pair remains the compact evidence-facing content |
| 188 | authority, not a locked presentation contract. Default Generate has Strategist |
| 189 | read both files completely before confirmation and use them with the imported |
| 190 | source inventory to select the content, page roster, and image resource plan. |
| 191 | Quick Generate has the current agent do the same before its active-context |
| 192 | content, design, and resource decisions. A webpage Markdown enters the project |
| 193 | only through the post-exhaustion single-page image fallback above. |
| 194 | |
| 195 | ```markdown |
| 196 | ## ✅ Topic Research Complete |
| 197 | - [x] Research execution: <isolated worker | main-context fallback> |
| 198 | - [x] Research supplement: `projects/<research_slug>.md` (N declared gaps covered) |
| 199 | - [x] Fact provenance: `projects/<research_slug>.facts.json` (N external facts) |
| 200 | - [x] Artifact contract validated: `## Research Brief`, no Markdown source list, `ppt-master.fact-provenance.v1`, unique sequential IDs, and Markdown/JSON agreement |
| 201 | - [x] Adopted webpage URLs: N unique `source_url` values in the facts JSON; no webpage automatically imported and no image copied into the runtime pool |
| 202 | - [ ] **Next**: Default returns to [`generate-pptx`](../generate-pptx.md) Step 2; Quick returns to [`quick-generate`](../profiles/quick-generate.md) §2. Import the source artifacts plus research pair, then fully read the imported pair before planning or direct SVG authoring |
| 203 | ``` |
| 204 |