| 1 | --- |
| 2 | name: crawling-social-media |
| 3 | description: Downloads and extracts content from social media platforms. Use when user wants to crawl, download, or extract videos/content from Bilibili, YouTube, TikTok, Douyin, Instagram, Twitter, Xiaohongshu. 下载视频、抓取视频、爬取内容、提取视频、下载B站视频、下载抖音视频、下载YouTube视频、视频采集。 |
| 4 | --- |
| 5 | |
| 6 | # Social Media Crawling |
| 7 | |
| 8 | Downloads and extracts content from social media platforms. |
| 9 | |
| 10 | ## Supported Platforms and URL Patterns |
| 11 | |
| 12 | | Platform | URL Pattern Examples | |
| 13 | | ----------- | ---------------------------------------------------- | |
| 14 | | Bilibili | `bilibili.com/video/BVxxxx`, `b23.tv/xxxxx` | |
| 15 | | YouTube | `youtube.com/watch?v=xxx`, `youtu.be/xxx` | |
| 16 | | TikTok | `tiktok.com/@user/video/xxx`, `vm.tiktok.com/xxx` | |
| 17 | | Douyin | `douyin.com/video/xxx`, `v.douyin.com/xxx` | |
| 18 | | Xiaohongshu | `xiaohongshu.com/explore/xxx`, `xhslink.com/xxx` | |
| 19 | | Kuaishou | `kuaishou.com/short-video/xxx`, `v.kuaishou.com/xxx` | |
| 20 | |
| 21 | ## Supported Content Types |
| 22 | |
| 23 | - **Videos**: Short videos, long videos from all platforms |
| 24 | - **Images**: Xiaohongshu image posts, Douyin image posts |
| 25 | - **Metadata**: Title, description, tags, cover image |
| 26 | |
| 27 | ## When to Use |
| 28 | |
| 29 | Use this skill when user: |
| 30 | |
| 31 | - Provides a social media URL matching the patterns above |
| 32 | - Wants to download/save videos from social platforms |
| 33 | - Wants to extract content for re-publishing |
| 34 | - Needs video metadata (title, description, tags) |
| 35 | - Mentions keywords like: download, save, crawl, extract, 下载, 抓取, 爬取 |
| 36 | |
| 37 | **Do NOT use** when: |
| 38 | |
| 39 | - User only wants to view/preview content (no download needed) |
| 40 | - URL is not from a supported platform |
| 41 | |
| 42 | ## Workflow |
| 43 | |
| 44 | ### Step 1: Get Social Media URL |
| 45 | |
| 46 | Obtain the post URL from user. |
| 47 | |
| 48 | ### Step 2: Submit Crawl Task |
| 49 | |
| 50 | Call `createCrawlTask` with the link. |
| 51 | |
| 52 | ### Step 3: Poll for Results |
| 53 | |
| 54 | 1. Wait **30 seconds** between status checks |
| 55 | 2. Call `getCrawlTaskStatus` to check progress |
| 56 | |
| 57 | ### Step 4: Return Results |
| 58 | |
| 59 | On success, return: |
| 60 | |
| 61 | - Video/media URLs |
| 62 | - Title |
| 63 | - Description |
| 64 | - Tags |
| 65 | |
| 66 | ## Examples |
| 67 | |
| 68 | ### Example 1: Download Bilibili Video |
| 69 | |
| 70 | **User request**: "Download this Bilibili video: https://www.bilibili.com/video/BV1xx..." |
| 71 | |
| 72 | ``` |
| 73 | 1. createCrawlTask: |
| 74 | - link: "https://www.bilibili.com/video/BV1xx..." |
| 75 | 2. Wait 30 seconds |
| 76 | 3. Poll getCrawlTaskStatus until success |
| 77 | 4. Return video URL, title, description, tags |
| 78 | ``` |
| 79 | |
| 80 | ### Example 2: Download from Short Link |
| 81 | |
| 82 | **User request**: "下载这个视频 https://v.douyin.com/xxx" |
| 83 | |
| 84 | ``` |
| 85 | 1. createCrawlTask: |
| 86 | - link: "https://v.douyin.com/xxx" |
| 87 | 2. Wait 30 seconds |
| 88 | 3. Poll getCrawlTaskStatus until success |
| 89 | 4. Return video URL, title, description, tags |
| 90 | ``` |
| 91 | |
| 92 | ### Example 3: Extract for Re-publishing |
| 93 | |
| 94 | ``` |
| 95 | 1. createCrawlTask with source URL |
| 96 | 2. Poll getCrawlTaskStatus until completed |
| 97 | 3. Extract media URLs, title, description, tags |
| 98 | 4. Use content skill to save media |
| 99 | 5. Use publish skill to publish to target platforms |
| 100 | ``` |
| 101 | |
| 102 | ## Task Status Values |
| 103 | |
| 104 | | Status | Description | |
| 105 | | ------- | ------------------------------ | |
| 106 | | pending | Task created, waiting to start | |
| 107 | | running | Task is actively processing | |
| 108 | | success | Task completed successfully | |
| 109 | | failed | Task failed with error | |
| 110 | |
| 111 | ## Important Notes |
| 112 | |
| 113 | - Processing time varies by platform and content size |
| 114 | - Large videos may take longer to download |
| 115 | - Some platforms may have rate limits |
| 116 | - Media URLs are temporary - save to content library for permanent storage |
| 117 |