| 1 | --- |
| 2 | name: editing-videos |
| 3 | description: Video editing using Volcengine Track structure. Supports cutting, trimming, adding text, stickers, audio, filters, effects, transitions, multi-clip compositions, speed adjustment, watermark removal. 视频剪辑、裁剪视频、添加文字、添加水印、添加音频、视频滤镜、视频特效、视频转场、多片段拼接、调整速度、去水印。 |
| 4 | --- |
| 5 | |
| 6 | # Video Editing with Track Structure |
| 7 | |
| 8 | Performs video editing using the Volcengine cloud editing API with native Track structure. |
| 9 | |
| 10 | ## ⚠️ CRITICAL: ALL PosX/PosY Are Top-Left Coordinates |
| 11 | |
| 12 | **ALL PosX/PosY values in this API are TOP-LEFT corner coordinates, NOT center point!** |
| 13 | |
| 14 | This applies to ALL filters: transform, crop, delogo, etc. |
| 15 | |
| 16 | **For full-screen video display, you MUST use:** |
| 17 | ```json |
| 18 | { "Type": "transform", "PosX": 0, "PosY": 0, "Width": <canvas_width>, "Height": <canvas_height> } |
| 19 | ``` |
| 20 | |
| 21 | **NEVER use canvas center coordinates (like 640, 360) for PosX/PosY!** |
| 22 | - Using center coordinates will cause video to display off-canvas |
| 23 | - The coordinate (0, 0) is the top-left corner of the canvas |
| 24 | |
| 25 | ## When to Use |
| 26 | |
| 27 | Use this skill when: |
| 28 | |
| 29 | - Cutting or trimming video segments |
| 30 | - Adding text, watermarks, or stickers to videos |
| 31 | - Adding or replacing audio tracks |
| 32 | - Applying filters or visual effects |
| 33 | - Merging multiple video clips |
| 34 | - Adding transitions between clips |
| 35 | - Changing video playback speed |
| 36 | - Removing watermarks (logo blur) |
| 37 | |
| 38 | ## Workflow |
| 39 | |
| 40 | ### Step 1: Get Video Information (Required) |
| 41 | |
| 42 | **ALWAYS call `uploadAndGetVid` first** to: |
| 43 | |
| 44 | 1. Upload video to Volcengine cloud |
| 45 | 2. Get VID for video source reference (format: `vid://xxx`) |
| 46 | 3. Get Duration for time calculations (in seconds) |
| 47 | 4. Get Dimensions (width, height) for Canvas setup |
| 48 | |
| 49 | If you only need metadata without editing (e.g., for display purposes), use `probeVideoMetadata` instead. |
| 50 | |
| 51 | ### Step 2: Build EditParam |
| 52 | |
| 53 | Construct your `EditParam` following the structure below. |
| 54 | |
| 55 | ### Step 3: Submit Task |
| 56 | |
| 57 | Call `submitDirectEditTask` with Canvas, Output (optional), and Track. |
| 58 | |
| 59 | ### Step 4: Poll for Results |
| 60 | |
| 61 | 1. First check: Wait **90 seconds** after submission |
| 62 | 2. Subsequent checks: Wait **30 seconds** between polls |
| 63 | 3. Call `getVideoEditTaskStatus` to check progress |
| 64 | 4. **Maximum wait time: 20 minutes** |
| 65 | |
| 66 | ### Step 5: Return Results |
| 67 | |
| 68 | On completion, return the output video URL to user. |
| 69 | |
| 70 | ## EditParam Structure |
| 71 | |
| 72 | ```json |
| 73 | { |
| 74 | "Canvas": { "Width": 1920, "Height": 1080 }, |
| 75 | "Output": { "Fps": 30, "Codec": { "VideoCodec": "h264" } }, |
| 76 | "Track": [ |
| 77 | [{ "Type": "video", "Source": "vid://xxx", "TargetTime": [0, 5000] }], |
| 78 | [{ "Type": "text", "Text": "Hello", "TargetTime": [0, 5000], "Extra": [] }] |
| 79 | ] |
| 80 | } |
| 81 | ``` |
| 82 | |
| 83 | **Canvas** (optional): |
| 84 | - If omitted, auto-detected from the primary video source dimensions |
| 85 | - If provided manually, MUST match getVideoInfo/uploadAndGetVid output |
| 86 | - Width = horizontal pixels, Height = vertical pixels |
| 87 | - Example: 1280x720 video → { Width: 1280, Height: 720 } — NOT { Width: 720, Height: 1280 } |
| 88 | ``` |
| 89 | |
| 90 | **Time unit**: All time values are in **MILLISECONDS** (1 second = 1000 ms) |
| 91 | |
| 92 | ## Track Structure |
| 93 | |
| 94 | Track is a **2D array** `Track[layerIndex][elementIndex]`: |
| 95 | |
| 96 | - Outer array: Rendering layers (higher index = higher layer, renders on top) |
| 97 | - Inner array: Elements on the same track |
| 98 | |
| 99 | ## Element Types |
| 100 | |
| 101 | ### video |
| 102 | |
| 103 | ```json |
| 104 | { |
| 105 | "Type": "video", |
| 106 | "Source": "vid://your_vid", |
| 107 | "TargetTime": [0, 5000], |
| 108 | "Extra": [ |
| 109 | { "Type": "trim", "StartTime": 10000, "EndTime": 30000 } |
| 110 | ] |
| 111 | } |
| 112 | ``` |
| 113 | |
| 114 | ### audio |
| 115 | |
| 116 | ```json |
| 117 | { |
| 118 | "Type": "audio", |
| 119 | "Source": "vid://your_audio_vid", |
| 120 | "TargetTime": [0, 10000], |
| 121 | "Extra": [ |
| 122 | { "Type": "a_volume", "Volume": 0.5 } |
| 123 | ] |
| 124 | } |
| 125 | ``` |
| 126 | |
| 127 | **When to use audio elements**: |
| 128 | |
| 129 | - Adding background music to video |
| 130 | - Adding voiceover/narration |
| 131 | - Adjusting audio volume separately from video |
| 132 | - Replacing original audio with different audio |
| 133 | |
| 134 | **When NOT to use audio elements**: |
| 135 | |
| 136 | - Simple video concatenation - video elements already contain their audio tracks |
| 137 | - When you want to keep the original video audio unchanged |
| 138 | |
| 139 | ### image |
| 140 | |
| 141 | ```json |
| 142 | { |
| 143 | "Type": "image", |
| 144 | "Source": "mid://your_image_mid", |
| 145 | "TargetTime": [0, 3000], |
| 146 | "Extra": [ |
| 147 | { "Type": "transform", "PosX": 100, "PosY": 100, "Width": 200, "Height": 200 } |
| 148 | ] |
| 149 | } |
| 150 | ``` |
| 151 | |
| 152 | ### text |
| 153 | |
| 154 | ```json |
| 155 | { |
| 156 | "Type": "text", |
| 157 | "Text": "Your Text Here", |
| 158 | "TargetTime": [0, 5000], |
| 159 | "FontType": "SY_Black", |
| 160 | "FontSize": 120, |
| 161 | "FontColor": "#FFFFFFFF", |
| 162 | "AlignType": 1, |
| 163 | "Extra": [ |
| 164 | { "Type": "transform", "PosX": 60, "PosY": 440, "Width": 1800, "Height": 200 } |
| 165 | ] |
| 166 | } |
| 167 | ``` |
| 168 | |
| 169 | ### subtitle |
| 170 | |
| 171 | ```json |
| 172 | { |
| 173 | "Type": "subtitle", |
| 174 | "Text": "http://example.com/subtitle.srt", |
| 175 | "TargetTime": [0, 20000], |
| 176 | "FontType": "ALi_PuHui", |
| 177 | "FontSize": 60, |
| 178 | "Extra": [ |
| 179 | { "Type": "transform", "PosX": 40, "PosY": 550, "Width": 1200, "Height": 150 } |
| 180 | ] |
| 181 | } |
| 182 | ``` |
| 183 | |
| 184 | ## Filter Types (Extra) |
| 185 | |
| 186 | ### transform - 2D Transform |
| 187 | |
| 188 | Controls element position, size, rotation, and opacity on the canvas. |
| 189 | |
| 190 | **IMPORTANT**: |
| 191 | |
| 192 | - `PosX` and `PosY` are the **TOP-LEFT corner** coordinates of the element relative to the canvas top-left corner |
| 193 | - They are NOT center point coordinates |
| 194 | - Rotation is performed around the element's center point |
| 195 | |
| 196 | **Parameters**: |
| 197 | |
| 198 | - `PosX`: X coordinate of element's **top-left corner** (pixels) |
| 199 | - `PosY`: Y coordinate of element's **top-left corner** (pixels) |
| 200 | - `Width`: Element width on canvas (pixels) |
| 201 | - `Height`: Element height on canvas (pixels) |
| 202 | - `Rotation`: Rotation angle [-360, 360], clockwise is positive (optional) |
| 203 | - `Alpha`: Opacity [0,1], 0 is transparent (optional) |
| 204 | |
| 205 | **Basic Example** (fill canvas): |
| 206 | |
| 207 | ```json |
| 208 | { "Type": "transform", "PosX": 0, "PosY": 0, "Width": 1920, "Height": 1080 } |
| 209 | ``` |
| 210 | |
| 211 | **Rotation Example** (rotate 1280x720 video 90° to fit 720x1280 canvas): |
| 212 | |
| 213 | When rotating, the element rotates around its center. To fill a 720x1280 canvas with a 1280x720 video rotated 90°: |
| 214 | |
| 215 | - After rotation, the video's display size becomes 720x1280 |
| 216 | - The center should be at canvas center (360, 640) |
| 217 | - Calculate: PosX = 360 - 1280/2 = -280, PosY = 640 - 720/2 = 280 |
| 218 | |
| 219 | ```json |
| 220 | { "Type": "transform", "PosX": -280, "PosY": 280, "Width": 1280, "Height": 720, "Rotation": 90 } |
| 221 | ``` |
| 222 | |
| 223 | **Centering Formula**: |
| 224 | |
| 225 | - To center an element horizontally: `PosX = (CanvasWidth - ElementWidth) / 2` |
| 226 | - To center an element vertically: `PosY = (CanvasHeight - ElementHeight) / 2` |
| 227 | |
| 228 | Example: Center a 600x60 subtitle on 720x1280 canvas: |
| 229 | |
| 230 | - PosX = (720 - 600) / 2 = 60 |
| 231 | - PosY = 1280 - 100 = 1180 (near bottom) |
| 232 | |
| 233 | **Common Mistakes**: |
| 234 | |
| 235 | - WRONG: Using canvas center (360, 640) or (640, 360) as PosX/PosY - this places the element's top-left at center, causing it to go off-canvas |
| 236 | - WRONG: Omitting transform for full-screen video elements - video may not display correctly without explicit positioning |
| 237 | - WRONG: Creating separate audio tracks for each video clip in simple concatenation - video elements already contain their audio |
| 238 | - CORRECT: Use (0, 0) for top-left alignment when video should fill the canvas |
| 239 | - CORRECT: Only add audio tracks when you need background music, voiceover, or volume adjustment |
| 240 | |
| 241 | ### trim - Time Trim |
| 242 | |
| 243 | ```json |
| 244 | { "Type": "trim", "StartTime": 5000, "EndTime": 15000 } |
| 245 | ``` |
| 246 | |
| 247 | ### crop - Area Crop |
| 248 | |
| 249 | ```json |
| 250 | { "Type": "crop", "PosX": 100, "PosY": 100, "Width": 800, "Height": 600 } |
| 251 | ``` |
| 252 | |
| 253 | ### speed - Playback Speed |
| 254 | |
| 255 | ```json |
| 256 | { "Type": "speed", "Speed": 2.0 } |
| 257 | ``` |
| 258 | |
| 259 | ### transition - Transition Effect |
| 260 | |
| 261 | ```json |
| 262 | { "Type": "transition", "Source": "1182376", "Duration": 1000 } |
| 263 | ``` |
| 264 | |
| 265 | ### lut_filter - Color Filter |
| 266 | |
| 267 | ```json |
| 268 | { "Type": "lut_filter", "TargetTime": [0, 5000], "Source": "1184003", "Intensity": 0.8 } |
| 269 | ``` |
| 270 | |
| 271 | ### video_animation - Video Animation |
| 272 | |
| 273 | ```json |
| 274 | { "Type": "video_animation", "AnimRes": "1180337", "AnimStartTime": 0, "AnimEndTime": 1000 } |
| 275 | ``` |
| 276 | |
| 277 | ### a_volume - Audio Volume |
| 278 | |
| 279 | ```json |
| 280 | { "Type": "a_volume", "Volume": 0.5 } |
| 281 | ``` |
| 282 | |
| 283 | ### a_fade - Audio Fade |
| 284 | |
| 285 | ```json |
| 286 | { "Type": "a_fade", "FadeIn": 2000, "FadeOut": 2000 } |
| 287 | ``` |
| 288 | |
| 289 | ### delogo - Logo Blur |
| 290 | |
| 291 | ```json |
| 292 | { "Type": "delogo", "TargetTime": [0, 10000], "PosX": 1700, "PosY": 50, "Width": 200, "Height": 80, "Sigma": 30, "Radius": 30 } |
| 293 | ``` |
| 294 | |
| 295 | ## Resource IDs |
| 296 | |
| 297 | ### Filter IDs (lut_filter) |
| 298 | |
| 299 | | ID | Name | Description | |
| 300 | | ------- | --------- | ----------- | |
| 301 | | 1184003 | Clear | Enhanced clarity | |
| 302 | | 1184004 | Afternoon | Warm afternoon tone | |
| 303 | | 1183995 | Vintage | Retro film look | |
| 304 | | 1183993 | Friends | Friends TV show style | |
| 305 | |
| 306 | ### Transition IDs |
| 307 | |
| 308 | | ID | Name | Description | |
| 309 | | ------- | ---------- | ----------- | |
| 310 | | 1182376 | CircleOpen | Circle wipe open | |
| 311 | | 1182360 | RotateZoom | Rotate and zoom | |
| 312 | | 1182370 | DoorOpen | Door opening reveal | |
| 313 | | 1182379 | ClockWipe | Clock sweep wipe | |
| 314 | |
| 315 | ### Video Animation IDs |
| 316 | |
| 317 | | ID | Name | Type | |
| 318 | | ------- | ------- | ---- | |
| 319 | | 1180337 | FadeIn | In | |
| 320 | | 1180382 | FadeOut | Out | |
| 321 | | 1180335 | Shrink | In | |
| 322 | | 1180338 | ZoomIn | In | |
| 323 | |
| 324 | ### Font IDs |
| 325 | |
| 326 | | ID | Name | |
| 327 | | ----------- | -------------- | |
| 328 | | SY_Black | Source Han Sans Black | |
| 329 | | ALi_PuHui | Alibaba PuHuiTi | |
| 330 | | PM_ZhengDao | Pangmen Zhengdao Title | |
| 331 | |
| 332 | ## Examples |
| 333 | |
| 334 | ### Example 1: Simple Video Concatenation (Most Common) |
| 335 | |
| 336 | ```json |
| 337 | { |
| 338 | "Canvas": { "Width": 1920, "Height": 1080 }, |
| 339 | "Track": [[ |
| 340 | { |
| 341 | "Type": "video", |
| 342 | "Source": "vid://video1", |
| 343 | "TargetTime": [0, 5000], |
| 344 | "Extra": [ |
| 345 | { "Type": "transform", "PosX": 0, "PosY": 0, "Width": 1920, "Height": 1080 } |
| 346 | ] |
| 347 | }, |
| 348 | { |
| 349 | "Type": "video", |
| 350 | "Source": "vid://video2", |
| 351 | "TargetTime": [5000, 12000], |
| 352 | "Extra": [ |
| 353 | { "Type": "transform", "PosX": 0, "PosY": 0, "Width": 1920, "Height": 1080 } |
| 354 | ] |
| 355 | }, |
| 356 | { |
| 357 | "Type": "video", |
| 358 | "Source": "vid://video3", |
| 359 | "TargetTime": [12000, 20000], |
| 360 | "Extra": [ |
| 361 | { "Type": "transform", "PosX": 0, "PosY": 0, "Width": 1920, "Height": 1080 } |
| 362 | ] |
| 363 | } |
| 364 | ]] |
| 365 | } |
| 366 | ``` |
| 367 | |
| 368 | **Key points for video concatenation**: |
| 369 | |
| 370 | - Transform with `PosX: 0, PosY: 0` is REQUIRED for full-screen display |
| 371 | - NO separate audio tracks - video elements already contain their audio |
| 372 | - All video clips on the same track (single inner array) |
| 373 | - Sequential TargetTime: video2 starts where video1 ends |
| 374 | |
| 375 | ### Example 2: Concatenate Videos with Transition |
| 376 | |
| 377 | ```json |
| 378 | { |
| 379 | "Canvas": { "Width": 1920, "Height": 1080 }, |
| 380 | "Track": [[ |
| 381 | { |
| 382 | "Type": "video", |
| 383 | "Source": "vid://video1", |
| 384 | "TargetTime": [0, 5000], |
| 385 | "Extra": [ |
| 386 | { "Type": "transform", "PosX": 0, "PosY": 0, "Width": 1920, "Height": 1080 }, |
| 387 | { "Type": "transition", "Source": "1182376", "Duration": 1000 } |
| 388 | ] |
| 389 | }, |
| 390 | { |
| 391 | "Type": "video", |
| 392 | "Source": "vid://video2", |
| 393 | "TargetTime": [4000, 9000], |
| 394 | "Extra": [ |
| 395 | { "Type": "transform", "PosX": 0, "PosY": 0, "Width": 1920, "Height": 1080 } |
| 396 | ] |
| 397 | } |
| 398 | ]] |
| 399 | } |
| 400 | ``` |
| 401 | |
| 402 | **Note**: With transitions, TargetTime values overlap (video2 starts at 4000, video1 ends at 5000). |
| 403 | |
| 404 | ### Example 3: Trim Video (10s to 30s) |
| 405 | |
| 406 | ```json |
| 407 | { |
| 408 | "Canvas": { "Width": 1920, "Height": 1080 }, |
| 409 | "Track": [[ |
| 410 | { |
| 411 | "Type": "video", |
| 412 | "Source": "vid://xxx", |
| 413 | "TargetTime": [0, 20000], |
| 414 | "Extra": [ |
| 415 | { "Type": "trim", "StartTime": 10000, "EndTime": 30000 } |
| 416 | ] |
| 417 | } |
| 418 | ]] |
| 419 | } |
| 420 | ``` |
| 421 | |
| 422 | ### Example 4: Add Text Watermark |
| 423 | |
| 424 | ```json |
| 425 | { |
| 426 | "Canvas": { "Width": 1920, "Height": 1080 }, |
| 427 | "Track": [ |
| 428 | [{ "Type": "video", "Source": "vid://xxx", "TargetTime": [0, 10000] }], |
| 429 | [{ |
| 430 | "Type": "text", |
| 431 | "Text": "@MyChannel", |
| 432 | "TargetTime": [0, 10000], |
| 433 | "FontType": "SY_Black", |
| 434 | "FontSize": 60, |
| 435 | "FontColor": "#FFFFFFFF", |
| 436 | "Extra": [ |
| 437 | { "Type": "transform", "PosX": 1700, "PosY": 50, "Width": 200, "Height": 60 } |
| 438 | ] |
| 439 | }] |
| 440 | ] |
| 441 | } |
| 442 | ``` |
| 443 | |
| 444 | ### Example 5: Add Background Music |
| 445 | |
| 446 | ```json |
| 447 | { |
| 448 | "Canvas": { "Width": 1920, "Height": 1080 }, |
| 449 | "Track": [ |
| 450 | [{ "Type": "video", "Source": "vid://video", "TargetTime": [0, 30000] }], |
| 451 | [{ |
| 452 | "Type": "audio", |
| 453 | "Source": "vid://music", |
| 454 | "TargetTime": [0, 30000], |
| 455 | "Extra": [ |
| 456 | { "Type": "a_volume", "Volume": 0.3 } |
| 457 | ] |
| 458 | }] |
| 459 | ] |
| 460 | } |
| 461 | ``` |
| 462 | |
| 463 | ### Example 6: Apply Filter |
| 464 | |
| 465 | ```json |
| 466 | { |
| 467 | "Canvas": { "Width": 1920, "Height": 1080 }, |
| 468 | "Track": [[ |
| 469 | { |
| 470 | "Type": "video", |
| 471 | "Source": "vid://xxx", |
| 472 | "TargetTime": [0, 10000], |
| 473 | "Extra": [ |
| 474 | { "Type": "lut_filter", "TargetTime": [0, 10000], "Source": "1183993", "Intensity": 0.8 } |
| 475 | ] |
| 476 | } |
| 477 | ]] |
| 478 | } |
| 479 | ``` |
| 480 | |
| 481 | ### Example 7: Logo Blur (Watermark Removal) |
| 482 | |
| 483 | ```json |
| 484 | { |
| 485 | "Canvas": { "Width": 1920, "Height": 1080 }, |
| 486 | "Track": [[ |
| 487 | { |
| 488 | "Type": "video", |
| 489 | "Source": "vid://xxx", |
| 490 | "TargetTime": [0, 10000], |
| 491 | "Extra": [ |
| 492 | { "Type": "delogo", "TargetTime": [0, 10000], "PosX": 1700, "PosY": 50, "Width": 200, "Height": 80, "Sigma": 30, "Radius": 30 } |
| 493 | ] |
| 494 | } |
| 495 | ]] |
| 496 | } |
| 497 | ``` |
| 498 | |
| 499 | ### Example 8: Rotate Landscape Video to Portrait |
| 500 | |
| 501 | Convert a 1280x720 (landscape) video to 720x1280 (portrait) canvas: |
| 502 | |
| 503 | ```json |
| 504 | { |
| 505 | "Canvas": { "Width": 720, "Height": 1280 }, |
| 506 | "Track": [[ |
| 507 | { |
| 508 | "Type": "video", |
| 509 | "Source": "vid://xxx", |
| 510 | "TargetTime": [0, 10000], |
| 511 | "Extra": [ |
| 512 | { |
| 513 | "Type": "transform", |
| 514 | "PosX": -280, |
| 515 | "PosY": 280, |
| 516 | "Width": 1280, |
| 517 | "Height": 720, |
| 518 | "Rotation": 90 |
| 519 | } |
| 520 | ] |
| 521 | } |
| 522 | ]] |
| 523 | } |
| 524 | ``` |
| 525 | |
| 526 | **Calculation**: |
| 527 | |
| 528 | - Original video: 1280x720, Canvas: 720x1280 |
| 529 | - After 90° rotation, video displays as 720x1280 |
| 530 | - Rotation center = (PosX + Width/2, PosY + Height/2) should equal canvas center (360, 640) |
| 531 | - PosX = 360 - 1280/2 = -280 |
| 532 | - PosY = 640 - 720/2 = 280 |
| 533 | |
| 534 | ## Important Notes |
| 535 | |
| 536 | - **Always call uploadAndGetVid first** - Required for VID and dimensions |
| 537 | - **Time unit is milliseconds** - 1 second = 1000 ms |
| 538 | - **Track layers render bottom to top** - Track[0] is bottom, Track[1] is above it |
| 539 | - **Transition duration must < clip duration** |
| 540 | - **Subtitle tracks must be separate** - Each subtitle occupies one track, max 10 subtitle tracks |
| 541 | - **Maximum processing time: 20 minutes** |
| 542 | - **Full-screen video requires transform with PosX: 0, PosY: 0** - Do NOT use canvas center coordinates like (640, 360) or (360, 640) |
| 543 | - **Video elements already contain audio** - Do NOT create separate audio tracks for simple video concatenation; only use audio tracks for background music, voiceover, or volume adjustment |
| 544 | - **Canvas is optional** — omit it for auto-detection from video sources; only specify manually for custom canvas sizes |
| 545 |