# VlogMe Script Grammar — canonical contract > Single source of truth for two contracts: > > **A. Frontend ↔ Backend (script grammar)** — what humans and AI write. > **B. Create ↔ Render domain (payload)** — the structured project scenes the > V2 preparation pipeline consumes. > > The frontend never builds provider payloads directly. `flatToPayloads()` in > `src/domains/create/scenes/flat-script.ts` is the single bridge from authored > grammar to `ScenePayload[]`. --- ## A. Script grammar (frontend ↔ backend) A VlogMe script is plain text driving a short-form render. The project-level `aspect_ratio` decides the frame (`9:16`, `16:9`, or `1:1`); the script grammar is identical for all three formats. There are **four explicit beat forms**, plus a plain-text continuation line for the current avatar. The shape of the token decides the kind — never the card role. | # | Form | Meaning | | --- | ---------------------- | ---------------------------------------------------------------------- | | 1 | `@imageN ` | Avatar speaks ``, anchored to photo slot N. | | 2 | `@imageN { }` | Standalone video clip generated from photo N. Prompt is in the braces. | | 3 | `{ @imageM }` | Overlay clip on top of the current avatar, using photo M as 1st frame. | | 4 | `{ }` | Continue — extends the previous clip, no reference image of its own. | Plus one media token: `@audioN` on its own line drops audio clip N on the current avatar. Plain text on its own line continues speech for the current avatar. This is mainly used after an overlay beat: ```text @image1 So I open the laptop. { @image3 slow push-in on the glowing keyboard }:3 And the cursor starts moving on its own. ``` ### Rules - Whitespace, blank lines and indentation are **insignificant** — only the token shape matters. - Avatar text (form 1) preserves its newlines verbatim into the scene payload's `script` field. Multi-line dialogue stays multi-line end-to-end. - The current avatar = the most recent avatar speech tag. Forms 3 and 4 anchor to that avatar/beat. - Form 4 requires a preceding rendered frame/current avatar to continue from. - Form 3 starts directly with `@imageN`: `{ @image3 fade out }`. - In form 4, the optional keyword `continue` is stripped: `{ continue sparks fly }` works the same as `{ sparks fly }`. - `@imageN-K` (K in 0..100): cross-fade transition code applied when the avatar switches (form 1 only). ### Duration override Any form 2 / 3 / 4 may carry an explicit clip duration in seconds: - `{ ... }:D` — after the closing brace. - `@imageN { ... }:D` — same. Without `:D` the clip uses the system default (3s), capped at 5s. ### Brace body options Forms 2 / 3 / 4 can use either a plain prompt or an advanced pipe body: ```text @image2 { slow push-in on a mountain lake }:3 { @image3 v:glowing OPEN sign, slow push-in | n:watermark, text | s:soft city hum | t5 }:3 { v:camera keeps drifting upward | am:off }:2 ``` Supported segments: - `v:` — main visual prompt. - `n:` — visual negative prompt. - `s:` / `an:` — generated audio prompt fields. - `@audioM` — uploaded audio clip under this video beat. - `am:auto|prompt|asset|off` — audio mode. - `ag:` — audio gain. - `tK` — transition code, 0..100. Generation prompt fields accept up to 10,000 characters. A selected provider may impose a lower native limit; Video Studio validates those provider-specific limits before submission. ### Examples ```text @image1 Hey, let me show you something. @image1 So I open the laptop. { @image3 slow push-in on the glowing keyboard }:3 And the cursor starts moving on its own. @image2 { dolphin breaches in slow motion against the sunrise }:4 { camera keeps drifting up, sky turns deeper orange }:2 @image1 [shocked] What the hell is happening? @audio1 ``` Breakdown: - Line 1 — avatar 1 speaks one sentence. - Line 3 — overlay over the current avatar using photo 3. - Line 4 — avatar 1 continues speaking. - Line 6 — standalone video from photo 2 (4s). - Line 7 — continue clip extending the previous shot (2s, no reference photo). - Line 9 — avatar 1 again, with an ElevenLabs `[shocked]` audio tag. - Line 10 — drops audio clip 1 on top of the avatar. ### Legacy back-compat The old "bare `@imageN` lets the card's role decide" form is still accepted by the parser. Specifically: an `@imageN` line **without** braces falls back to `card.role` + `card.effectInsertMode` to pick avatar / video / overlay / continue. This keeps the timeline editor — which currently edits cards in the bare form — working unchanged while we migrate the UI to render explicit brace cards. When AI generates a script, it should prefer the **four explicit forms**. Parser/serializer round-trips currently normalize back to the legacy bare form for the timeline UI; that will change in a follow-up. ### Forbidden - Mid-line `@imageN` outside of a brace. Each tag is on its own line OR inside a `{ … }` block. - Inline overlay inside avatar speech, e.g. `@image1 words {@image2 ...}:3`. - Bare `@imageN` video instructions from new AI output. Generate `@imageN { ... }:D` instead. - Nested braces. - Anything else not listed in §A. --- ## B. Project scene payload (Create ↔ Render domain) The V2 preparation pipeline consumes `ScenePayload[]`, defined by the public `@/domains/create` contract. `flatToPayloads()` is the only producer from the flat grammar; provider-specific payloads are derived later inside render and integration infrastructure. Each scene has an `input_type` discriminator: | `input_type` | Comes from script form | Key fields | | --------------- | ---------------------- | ---------------------------------------------------------------------------------------------------------------------------- | | `text` | Form 1 | `image_path`, `script` (avatar speech), `voice_id`, `transition` | | `audio` | `@audioN` | `image_path` (current avatar), `audio_path`, `audio_duration_seconds` | | `ltx` | Form 2 | `image_path` (source photo), `script` (visual prompt), `audio_duration_seconds` | | `video_overlay` | Form 3 | `image_path` (anchor avatar), `overlay_source_image_path` (photo M), `overlay_visual_prompt`, `overlay_min_duration_seconds` | | `video_chain` | Form 4 | `chain_from_previous: true`, `overlay_source_image_path`, `overlay_visual_prompt` | Insert variants (overlay / cut) for nested timing inside an avatar audio chunk follow `docs/public-api-inserts-contract.md`. That file documents the internal wire normalization used by V2 preparation and the MCP convenience adapter. The scene payload shape is **stable**. Map authored grammar in `flatToPayloads()` first; do not make users or AI write provider JSON. --- ## Pipeline (who owns what) ```text WishesField ← user types brief + uploads media │ ▼ buildScriptFromBrief (server) ← Gemini generates canonical script │ ▼ useFlatScriptBridge ← parses script, mirrors timeline ↔ Code │ ▼ TimelineBlocks ← user-editable cards │ ▼ flatToPayloads() ← THE bridge: script → ScenePayload[] │ ▼ immutable V2 project revision ← stores validated project scenes │ ▼ POST /api/v2/renders or MCP adapter ← durable job + credit reservation │ ▼ Cloudflare Worker + Queue ← leases and advances deterministic steps │ ▼ render infrastructure ← audio/video preparation and provider attempts │ ▼ API-only providers + owned storage ← Hedra / ModelArk / FAL / Replicate output ``` **Key principle**: the script grammar is the _only_ thing the user (or AI) authors. Everything downstream — scene shape, provider inputs, asset paths, voice IDs, transition codes — is derived. Adding new script behavior starts in `flatToPayloads()`; don't ask AI or users to write payload JSON.