NexArtWorkFlow

# NexArt Workflow Studio <p align="center"> <img src="./docs/assets/nexart-workflow-hero.svg" alt="NexArt Workflow Studio hero" width="100%" /> </p> <p align="center"> Visual Dream Creation Flow Platform: Put reference images, prompts, chain iterations, batch generation, and MCP calls on the same canvas. </p> <p align="center"> <code>Dreamina CLI</code> · <code>React Flow</code> · <code>Chain Run</code> · <code>Image / Video</code> · <code>MCP</code> </p> <p align="center"> <a href="./CHANGELOG.md">Release Notes</a> · <a href="#-quick-start">Quick Start</a> · <a href="#-architecture">Architecture</a> · <a href="#-api-reference">API</a> </p> > Latest Release: Added Gemini API image/video generation, Append video stitching node, long takeover default strategy, video duration display, and production deployment unified with `dist/server.cjs`. Details see `CHANGELOG.md`. ``` ┌────────┐ ┌──────────┐ ┌────────┐ │ Input │→│ Prompt │→│ Output │ Each Prompt node submits N parallel tasks │ (n1.png)│ │ @n1 改成... │ │ (1 image) │ Downstream Prompt automatically waits for upstream Output to complete └────────┘ └──────────┘ └────────┘ ``` --- ## ✨ Features ### Canvas Editing - **Visual Nodes**: Input (reference image/video) / Prompt (prompt + parameters) / Output (generated result) / Append (video stitching) / Group - **Drag-and-Drop Upload**: Drag images directly to the canvas, automatically upload to local data directory, and InputNode naming takes from file name (`n1.png` → `n1`) - **@-token Reference**: Write `@n1` / `@output1` in prompt to reference upstream Input / Output's local file, automatically as dreamina's image input - **Independent Node Dragging**: Each node moves separately, without child nodes following - **L Automatic Layout**: dagre layouts according to real node size, large nodes do not overlap - **Multi-Selection Interaction**: `Shift + left click` to select multiple nodes, `Ctrl/Cmd + left click` to deselect nodes - **Output Node Coloring**: Input keeps green, Output recovers yellow; image Output and video Output use different yellow shades to distinguish - **Video Duration Visible**: Video Output node displays duration label, still retained after refresh ### Dreamina CLI Integration - **Real Image/Video Generation** (not mock placeholder images) - **Supported Subcommands**: `text2image` / `image2image` / `text2video` / `image2video` (automatically selected based on upstream image) - **Image Models**: `seedream 5.0` (locked, latest dreamina) - **Video Models**: `seedance2.0fast` (default) / `seedance2.0` / `seedance2.0_vip` / `seedance2.0fast_vip` - **Aspect Ratios**: 8 types (1:1 / 4:3 / 3:2 / 16:9 / 21:9 / 3:4 / 2:3 / 9:16), `image2video` automatically infers from input image - **Generation Quantity**: Default 1 image (can be expanded to more; each independent submission, automatically creates corresponding number of OutputNode) - **Video Duration**: 4–15 seconds (seedance2.0 series) - **Long Video Patient Polling**: CLI `--poll` insufficient, backend continues `query_result` to make up, up to 30 minutes ### Gemini API Integration - **Image Generation**: Supports `Nano Banana 2` / `Nano Banana Pro` / `Nano Banana` - **Video Generation**: Supports `veo3.1` (frontend product name), backend automatically normalized to `veo-3.1-generate-preview` - **Video Aspect Ratio Limitation**: Gemini video only opens `16:9` / `9:16` - **Video Duration**: Gemini video supports `4s / 6s / 8s`; with reference image, automatically normalized to `8s` - **Reference Image Video**: Video API will pass reference image and `durationSeconds` to Google, not a placeholder switch ### Long Takeover / Video Stitching - **Long Takeover Default Strategy**: Video takeover prioritizes using upstream video tail `2–3s` short clip as reference, automatically compressed to `640px` upper limit - **Default Auto Tail Frame Disabled**: Dreamina multimodal video takeover defaults `autoTailFrames=off`, reducing face control trigger probability - **Explicit Role Image Enhancement**: If more stable character consistency is needed, prioritize additional connection role reference images, rather than automatically uploading tail frame static image - **Append Node**: Supports `video Output -> Append -> Output` stitching workflow, backend uses `ffmpeg concat` ### Workflow Orchestration - **▶ Run**: Run a single node - **Run Selected/All**: Batch submission, **divided into waves according to dependency graph** (B's upstream Output comes from A → B, etc., A completes), nodes without dependencies run in parallel - **Progress Floating Window**: Displays `task number / in progress / waiting / completed / failed` real-time statistics at the top center - **Result Immediate Display**: Optimistic update + backend write-back dual insurance, no longer stuck in progress mask ### Workflow Library - **Persistence**: Each workflow saved as `<dataDir>/workflows/<workflow-name>.json`, automatically loaded after server restart - **Automatic Old File Name Migration**: Old `1.json` / `<id>.json` will be migrated to the current workflow name corresponding JSON file during saving or loading - **Left Sidebar - "Workflow" Tab**: Lists all workflows, displays node count and JSON file name, click to switch, supports direct renaming within card, hover appears rename / copy / delete - **Import / Export**: Save As exports JSON, Open imports from JSON ### Product Archiving ``` ~/Documents/NexArtWorkFlow/outputs/ └── Movie-like Sci-Fi Scene/ ← workflow name └── 2026-05-24/ ← date └── 18-05-45_a3b7/ ← batch (HH-MM-SS_rand) ├── prompt1__output1.png ← <prompt name>__<output name>.ext ├── prompt1__output1.json ← complete metadata of the result ├── prompt1__prompt1_2.png ├── prompt1__prompt1_2.json ├── prompt2__output2.png ← second round of chained batch ├── references/ ← shared reference images for the entire batch │ └── n1.png └── batch.json ← batch index ``` ### MCP Interface (External Agent Call) - Exposes `http://localhost:3000/api/mcp/sse` to Claude Desktop / compatible MCP client - Tools: `get_workflow` / `add_flow_node` / `update_node_data` / `submit_jimeng_task` --- ## 🚀 Quick Start ### Prerequisites | Tool | Purpose | |---|---| | **Node.js** ≥ 18 | Run frontend and backend | | **dreamina CLI** | Real generation entrance | Install dreamina CLI (official ByteDance, installed to `~/.local/bin/dreamina`): ```bash curl -s https://jimeng.jianying.com/cli | bash dreamina login # OAuth device login, will give you a verification URL dreamina user_credit # Verify account + check remaining credit ``` ### Installation + Start ```bash git clone https://github.com/yaopengcheng11/NexArtWorkFlow.git cd NexArtWorkFlow npm install # Daily use / faster npm run build npm start # default: 3000, production mode # Local development / need hot update PORT=3001 npm run dev ``` Open http://localhost:3000 (or the port you set) in the browser. > On WSL, the first screen of Vite dev will be significantly slower on `/mnt/*` disk. For stability and speed, prioritize `npm run build && npm start`. ### Complete Generation 1. Drag a local image (e.g., `n1.png`) into the canvas → automatically create InputNode, named `n1` 2. Draw a line from `n1` to the automatically appearing PromptNode 3. Write prompt: `Change @n1 to watercolor style` 4. Select quantity (default 1 image) → canvas will automatically add corresponding number of OutputNode 5. Click PromptNode top-right **▶ Run** → top progress floating window displays task progress → results appear in OutputNode one by one 6. Drag a line from an OutputNode → new PromptNode, write prompt `@<output name>` to chain generate ### Video Stitching 1. Prepare two or more generated video Output nodes 2. Create an **Append** node, connect these video Outputs to Append left side 3. Connect from Append right side to a new Output node as result receiver 4. Click Append node top-right **Run** → backend calls `ffmpeg` stitching → result video writes back to downstream Output node ### Workflow Mode + Chained Batch 1. Build a chain on the canvas: `input → prompt1 → output1 → prompt2 → output2` 2. Top-left click **Run All** → system runs prompt1 first, waits for output1 image generation, then runs prompt2 3. Top progress floating window displays total tasks, in progress, waiting in real-time --- ## ⚙️ Configuration ### Data Directory Default: `~/Documents/NexArtWorkFlow/`, three subdirectories: | Directory | Content | |---|---| | `inputs/` | Temporary reference images/videos dragged and uploaded | | `outputs/` | Generated results (organized by `<workflow>/<date>/<batch>/`) | | `workflows/` | JSON files for each workflow, automatically loaded when server starts | **Modify Path**: Click top-right gear → top "Data Directory" input box fill new path → Apply. Also, modify `~/.nexart-workflow/config.json`'s `dataDir`. > ⚠️ Switching data directory will not automatically migrate old files, need to manually move them. ### Engine Mode PromptNode's "Engine" dropdown: - **Dreamina CLI**: Go to local `dreamina` command line, need OAuth login, support images and videos - **Gemini API**: Supports image and video generation, save `Gemini API Key` in top-right gear to use Gemini API current built-in image model aliases: - `Nano Banana 2` → `gemini-3.1-flash-image-preview` - `Nano Banana Pro` → `gemini-3-pro-image-preview` - `Nano Banana` → `gemini-2.5-flash-image` Gemini API current video models: - `veo3.1` → backend normalized to `veo-3.1-generate-preview` Gemini video constraints: - Aspect ratio only supports `16:9` / `9:16` - Duration supports `4s / 6s / 8s` - If current video Prompt node has reference image, duration automatically normalized to `8s` Gemini API Key saved locally: - `~/.nexart-workflow/config.json` The file is not in the repository and will not be included in git. ### Port `PORT=3002 npm run dev` to change port. Default 3000. --- ## 🏗️ Architecture ``` ┌─────────────────────────────────────────────────────────────┐ │ Browser (React 19 + xyflow + Tailwind v4) │ │ │ │ Workspace.tsx │ │ ├─ ReactFlow canvas (drag/drop/connection/batch selection/Append) │ │ ├─ Nodes: InputNode / PromptNode / OutputNode / AppendNode / GroupNode │ │ ├─ Left sidebar: canvas outline / workflow library / MCP │ │ ├─ Top progress floating window (subscribe task-start/done events) │ │ ├─ Top-left Run All / Run Selected (dependency graph scheduling) │ │ ├─ Settings (data directory configuration + Gemini API Key) │ └────────────────────────┬────────────────────────────────────┘ │ /api/run-jimeng, /api/upload, │ /api/workflow, /api/config, │ /api/mcp/sse, /api/mcp-logs ▼ ┌─────────────────────────────────────────────────────────────┐ │ Express server (dev: Vite middleware / prod: dist static resources)│ │ │ │ ├─ /api/run-jimeng → spawn dreamina <subcommand> --poll N │ │ │ long task timeout use query_result continue polling up to 30 minutes │ │ │ download remote result → outputs/<wf>/<date>/<batch>/<...>.png │ │ │ automatically write back outputNode (workflowState + version++) │ │ │ │ │ ├─ /api/upload → base64 → <dataDir>/inputs/<timestamp>_<name> │ │ ├─ /api/workflow GET/POST → memory + disk (each wf a .json)│ │ ├─ /api/workflows/:id DELETE │ │ ├─ /api/config GET/POST → ~/.nexart-workflow/config.json │ │ ├─ Gemini API → image/video generation + result save + write back OutputNode │ │ ├─ /api/append-video → ffmpeg concat + write back video Output │ │ ├─ /api/mcp (JSON-RPC) + /api/mcp/sse (SSE) → external Agent │ │ └─ static /outputs /inputs via dataDir wrapper, support hot switch │ └────────────────────────┬────────────────────────────────────┘ │ spawn child process ▼ dreamina CLI (~/.local/bin/dreamina) ├─ text2image / image2image ├─ text2video / image2video └─ query_result --submit_id ... ``` ### State Sync Model - Frontend holds local React state via `useNodesState` / `useEdgesState` - 500ms debounce to backend `POST /api/workflow` push (`version++`) - Backend in-memory `workflowState` simultaneously written by dreamina completion callback (`updateActiveTemplateNode` → `version++`) - Frontend polls `GET /api/workflow?version=N` every 1.5s, service version higher, pull new cover `setNodes` / `setEdges` - `isPollingPaused` temporarily (1.2s) turns off polling after push to avoid push-pull conflict - Key hooks: `task-start` / `task-done` / `batch-start` window events drive progress floating window; `prompt-run` / `prompt-done` give batch scheduler sync signal --- ## 🧱 Main Files | File | Function | |---|---| | `server.ts` | Express, `runJimengCLI`, Gemini, Append, MCP, workflow persistence, configuration | | `src/views/Workspace.tsx` | Canvas, batch scheduling, progress floating window, sidebar, settings panel | | `src/nodes/InputNode.tsx` | Reference image node + upload status | | `src/nodes/PromptNode.tsx` | Prompt node + run button + automatic Output supplement | | `src/nodes/OutputNode.tsx` | Result node + open folder + video duration display | | `src/nodes/AppendNode.tsx` | Video stitching node | | `src/nodes/GroupNode.tsx` | Node group container | | `src/i18n.tsx` | English/Chinese | --- ## 🔌 API Quick Check | Method | Endpoint | Description | |---|---|---| | `POST` | `/api/run-jimeng` | Submit a generation, body: `prompt / generationType / aspectRatio / model / numImages / outputNodeId / mode / images / batchId / promptName / outputName / duration` | | `POST` | `/api/append-video` | Concatenate multiple video inputs, write back a video output | | `POST` | `/api/upload` | Upload image in base64, return `{absPath, url}` | | `GET` `POST` | `/api/workflow` | Workflow status synchronization | | `DELETE` | `/api/workflows/:id` | Delete a workflow | | `GET` `POST` | `/api/config` | Data directory configuration | | `POST` | `/api/open-folder` | Open path in system file manager | | `GET` | `/api/mcp-logs` | Real-time log stream | | `POST` | `/api/mcp` + `GET` `/api/mcp/sse` | MCP JSON-RPC + SSE | --- ## 🤖 Connect Claude Desktop / MCP Client Paste this into the MCP configuration of the client: ```json { "transport": { "type": "sse", "url": "http://localhost:3000/api/mcp/sse" } } ``` Available tools: - `get_workflow` — Get the current canvas - `add_flow_node` — Add a node - `update_node_data` — Update node data - `submit_jimeng_task` — Let the agent run the generation for you --- ## 🛠️ Tech Stack - **Frontend**: React 19, TypeScript, Vite 6, Tailwind CSS v4, @xyflow/react 12, dagre, lucide-react, motion - **Backend**: Express 4, tsx, child_process to run dreamina - **Generation**: Official CLI of Jimeng (seedream for images / seedance for videos) - **Persistence**: Disk JSON (one file per workflow) --- ## 📝 Known Limitations - High-resolution long videos and multi-concurrent tasks still consume significant credits; monitor your account balance - Dreamina's long-shot continuation encounters clear faces and may trigger platform risk control, even with the short video strategy at the end - Append node currently only supports video output as input, not images or mixed media - No undo for bulk node deletion (currently relies on React Flow's built-in Cmd+Z) - Switching data directories does not automatically migrate old files --- ## 📄 License MIT (add LICENSE file for formal release)