Qwen-Image-2.1 REST Server

A minimal headless REST server for Qwen-Image-2.1 (unified text-to-image + image editing, 7B DiT) tuned for a single RTX 5070 (12 GB VRAM) + 46 GB RAM. Jobs run one at a time through a FIFO queue; results and job state are persisted on disk.

License note: Qwen-Image-2.1 weights are under the Qwen Research License (non-commercial). The Qwen3-VL-8B text encoder is Apache-2.0.

Layout

app/server.py        FastAPI app, GPU worker thread, /healthz
app/comfy_engine.py  ComfyUI engine (deployed): graphs, prompt tracking, timeouts
app/engine.py        diffusers engine (fallback; placement/quantization via env vars)
app/editing.py       edit preparation shared by both engines: mask modes, sizing, seeds
app/masking.py       rects, annotations, mask images, prompt composition,
                     composite-back + tone match, crop planning
app/imageio.py       upload normalisation, grey-in/grey-out output
app/jobstore.py      disk-backed job store + queue, partial outputs
app/params.py        request validation
app/errors.py        error codes of failed jobs
app/loras.py         LoRA registry for GET /v1/loras
app/rewriter.py      prompt rewriting via llama.cpp (mode "rewrite")
app/logconf.py       log filter/formatter; app/logging.json is the uvicorn log config
run_server.sh        launcher (the systemd unit runs it)
examples/manga_clean.py  the manga-cleaning recipe (client-side windows)
scripts/battery.py   GPU test battery through the REST API (run_battery.sh wraps it;
                     --dry-run is offline)
scripts/checks.py    result checks shared by battery.py and smoke.py
scripts/smoke.py     in-process diffusers-engine smoke test (needs the GPU to itself)
scripts/client_example.py  thin general-purpose client
scripts/make_test_page.py  builds the demo page in assets/
tests/               GPU-free test suites (fake ComfyUI, stub torch)
deploy/              systemd user units + deploy/README.md
docs/research/       the 2026-09 server review and the open-items list
assets/              demo manga page, its rects, the SFX font
loras/               LoRA files (gitignored)
jobs/                job data (inputs, outputs, job.json; gitignored)

Run

cd ~/qwen-image-server
./run_server.sh [port]          # default 8500; QIS_HOST overrides 0.0.0.0

run_server.sh sets the backend defaults below and starts uvicorn with --log-config app/logging.json (see "Logging"). A bare .venv/bin/uvicorn app.server:app --host 0.0.0.0 --port 8500 also works, but logs every status poll and files everything at one journal priority.

Two interchangeable backends (same API, same masking/compositing behavior):

QIS_BACKEND=comfy      # default in run_server.sh: headless ComfyUI on
                       # 127.0.0.1:8288 with comfy-kitchen int8 CUDA
                       # kernels — ~2x faster sampling. Needs the
                       # qwen-comfy-backend service running.
QIS_BACKEND=diffusers  # self-contained single process, step-level
                       # progress, disk embed cache; slower sampling.
                       # (app/server.py's default when the variable is unset)

Diffusers-backend knobs (defaults chosen for the 5070):

QIS_DIT=int8       # transformer: int8|fp8|bnb8|bnb4|bf16|bf16-group
QIS_TE=swap        # text encoder: swap (int8, RAM-parked, GPU during encode)
                   #   | cpu (full precision, computes on CPU — slow for edits)
                   #   | cuda (resident; needs a small DiT)
QIS_VAE=cuda       # vae: cuda (tiled 512/448) | cpu
QIS_PRELOAD=1      # load models at startup instead of first job
QIS_EMBED_CACHE=1  # cache text-only prompt embeddings on disk

Comfy-backend knobs: QIS_COMFY_URL (default http://127.0.0.1:8288), QIS_COMFY_DIT/TE/VAE filenames (default: the Comfy-Org int8_convrot set), QIS_COMFY_QUEUE_TIMEOUT (600 s a prompt may wait in ComfyUI's queue), QIS_COMFY_RUN_TIMEOUT (900 s, the floor of a prompt's execution timeout) and QIS_COMFY_RUN_TIMEOUT_MAX (3600 s, its ceiling); see "Timeouts and polling". The backend must be ComfyUI >= 0.37 (the server tracks prompts through its /api/jobs endpoints).

Other paths: QIS_JOBS_DIR (default jobs/), QIS_LORA_DIR (default loras/), QIS_LLAMA_DIR (default tools/llama/llama-b11120).

Logging

app/logging.json (loaded by run_server.sh, code in app/logconf.py):

• Access lines of successful status polls (GET /v1/jobs/{id} and /healthz, 2xx/304) are dropped. Submits, downloads, cancels, the job list and every non-2xx answer are still logged. QIS_LOG_POLLS=1 keeps the polls as well.
• Under systemd every line carries a journald priority prefix (error, warning, info, debug), so a traceback lands at "err" as a whole. The prefix is added only when stderr is the journal (JOURNAL_STREAM); QIS_LOG_JOURNAL=1 / 0 forces it on / off.

Warnings and errors only:

journalctl --user -u qwen-image-server -p warning

Deployment

Two systemd user services, qwen-comfy-backend and qwen-image-server (server on 0.0.0.0:8500). The unit files are versioned in deploy/systemd/. deploy/README.md has install, update and drift checks, why each non-default setting exists (--cache-lru 256, KillMode=mixed, ...) and the pieces that live outside the repo (the ComfyUI checkout, model symlinks, the LoRA link, llama.cpp and the rewriter GGUFs). Install:

cp deploy/systemd/*.service ~/.config/systemd/user/
systemctl --user daemon-reload
systemctl --user enable --now qwen-comfy-backend qwen-image-server
# start-on-boot additionally needs: sudo loginctl enable-linger $USER

The server has no authentication: anyone on the LAN can submit, cancel and download jobs and their inputs.

See BENCHMARKS.md for measured performance on the RTX 5070.

API

Create a job (multipart form; payload is a JSON string):

curl -s http://localhost:8500/v1/jobs \
  -F 'payload={"mode":"generate","prompt":"a red fox in the snow","steps":25,"seed":1}'

Edit with rect masks:

curl -s http://localhost:8500/v1/jobs \
  -F 'payload={
        "mode": "edit",
        "prompt": "",
        "steps": 25,
        "seed": 7,
        "mask_mode": "annotate",
        "composite_back": true,
        "rects": [{"x":48,"y":68,"w":414,"h":226, "label":"remove the text"}]
      }' \
  -F image=@page.png

For a manga-cleaning pipeline (Orihon), don't send whole pages with region_mode: "crop": cut one small window per group of at most 3 text boxes (2 when the group holds floating text) on the client, send each as a full-mode edit with an explicit seed and composite_back: false, and paste the raw render back client-side. examples/manga_clean.py is a single-pass reference client for this (not the lab's full recipe, which adds candidates, judging and retries); docs/research/2026-09-server-review/README.md has the evidence.

Then:

GET    /v1/jobs/{id}               status + progress, the job record (below)
GET    /v1/jobs/{id}/result        output file (?index=n for multiple outputs): PNG,
                                   JSON for a rewrite. Done jobs, and failed or
                                   canceled jobs with partial outputs; 409 otherwise
GET    /v1/jobs/{id}/inputs/{name} an uploaded input or a debug file (debug_files)
DELETE /v1/jobs/{id}               cancel (queued or running)
GET    /v1/jobs?limit=50           recent jobs, newest first: id, status, created_at,
                                   error, error_code, retryable, max_retries.
                                   limit is clamped to [1, 2000]
GET    /v1/loras                   LoRA files and their strength ranges (see "LoRAs")
GET    /v1/version                 the server build: release tag, exact id, features
GET    /healthz                    backend, VRAM, queue, running job, `features`

progress depends on the engine. The comfy engine reports phases only: {"phase": "backend sampling", "step": 0, "total": 0} ("crop 1/2" in crop mode). The diffusers engine reports steps: {"phase": "denoise", "step": 12, "total": 25}. Other phases: "starting", "waiting for backend" and "retrying after out of memory" (both while queued again), "freeing backend VRAM", "rewriting".

/healthz returns status ("ok" or "degraded"), backend, engine_loaded, waiting_for_backend, waiting_since (epoch seconds the current wait for the backend began, else null), running_job (the id the worker is processing, null while idle or waiting for the backend), queue_length (waiting jobs, the running one not counted), build (as GET /v1/version, below), models (once the engine is loaded: on the comfy backend dit / te / vae as {file, sha256, size}, the sha256 being the HF blob hash the backend's model symlinks resolve to; on the diffusers backend the repo, snapshot commit and quantisation modes) backend_identity (comfy backend: as run.backend), backend_presubmit_failures (losses before submit in a row over all jobs) and last_backend_error (at, job, submitted, error), and features. On the comfy backend also backend_up, backend_busy and vram_free_gb / vram_total_gb (see "Backend outages and restarts").

Version. GET /v1/version returns the build cheaply (no backend probe): version (the last release tag, e.g. v0.5.0), id (exactly the tag for a release; v0.5.0-3-g<commit> for a checkout past it; -dirty with local changes), release (true only for a clean release checkout), commit, dirty and features. Every response also carries the build id in an X-Server-Version header, and each job records it in run.build. CHANGELOG.md lists the client-visible changes per release.

features lists the optional API extensions a client can check before relying on them: reuse_encoding, encode_cache_report, prompt_rewrite, loras, sampler_select, run_record, error_codes, partial_results, tone_match, build_identity (build/models here and in run), version_endpoint (GET /v1/version, X-Server-Version). Changes without a feature flag are listed in CHANGELOG.md. A server without a payload field rejects it with 422 unknown fields, so check tone_match before sending that field.

Payload fields

field	default	notes
mode	generate	generate, edit (requires an image file) or rewrite (see "Prompt rewriting")
prompt	—	instruction; per-rect labels can replace it in edit mode (see "Prompt composition")
steps	40	1–100; 20–25 is the community sweet spot
seed	random	output i of a job with seed S is replayed alone by seed S + i, num_images: 1. A drawn seed is recorded in run.seed
width/height	auto	generate only, both or neither: explicit output size, each 256–3072, and width × height at most 2048 × 2048 (4.19 MP; 2464 × 1680 fits). The comfy engine floors each side to a multiple of 32 (1328 renders at 1312); run.render_size has the size used. Edits keep the input's aspect ratio; size them with output_resolution
output_resolution	1024	256–2048: target area (side length) when width/height omitted
negative_prompt + true_cfg_scale	off	CFG doubles compute; model is designed for cfg=1. true_cfg_scale 1–10
num_images	1	up to 4
rects	[]	{x,y,w,h,label} pixel rects on the base image
mask_mode	annotate	annotate (colored boxes drawn on image, at most 8 rects), mask (separate B/W mask image), paint (rects painted white)
region_mode	full	full = edit whole page; crop = crop-and-stitch around each rect group. Deprecated (frozen: still works, no more fixes; to be removed once clients cut their own windows): it merges overlapping windows, seeds windows by merge order and returns no raw render. Cut windows client-side instead (see above)
composite_back	true	paste the model's pixels into the rects and keep the original everywhere else. Pixels beyond rect + expand + 3 × feather (the feathered footprint; 2.5 × measured) are byte-identical to the input
tone_match	true	with composite_back: correct the pasted render's luma per level (clipped ±3 %, one factor per pixel) so its paper and light tone match the scan (see "How masking works"). Pixels beyond the footprint stay byte-identical. Turn it off for creative edits that change the tone around a rect. Advertised as tone_match
feather	4	sigma of the Gaussian blend, px, centred on the grown rect's edge
expand	8	grow rects before compositing, px. With 4 / 8 the whole rect is replaced; the old defaults (8 / 0) left faint ghosts of strokes near the rect edge
crop_context	1.0	context margin around rects in crop mode (× rect size)
crop_min_size	512	minimum crop window side
keep_debug	false	store annotated/mask intermediates + final prompt in the job dir
loras	[]	[{"name", "strength"}], up to 4, see "LoRAs" (comfy backend)
sampler / scheduler	euler / simple	any ComfyUI KSampler name (comfy backend). Samplers that evaluate the model 2 or 3 times per step (heun, dpmpp_2s_ancestral, dpmpp_sde, heunpp2, ...) cost 2× or 3× per step; no prompt may run longer than QIS_COMFY_RUN_TIMEOUT_MAX (1 h by default)
reuse_encoding	true	reuse a cached prompt/condition-image encoding when the inputs are identical; false forces a fresh encode (see below)

Reference images: attach as repeated ref files (up to 9 next to the base image). They are passed to the model as additional condition images. In mask_mode: "mask" the mask is the second condition image, so the first ref becomes the third image.

Types are strict: a JSON null means "not given" (the default applies), booleans must be true/false, integers may be sent as integral numbers or digit strings, and NaN/Infinity, fractional counts (steps, sizes, num_images, ...), unknown fields or rects with extra keys are rejected with HTTP 422. Fractional rect coordinates are rounded outward (the rect never shrinks) and a fractional seed is truncated. seed + num_images - 1 must stay below 2^63, so every replay seed is valid. Rects may reach past the image edge; only their part inside the image is used.

Uploads

Every uploaded image is normalised once, at submit time, to 8-bit RGB or 8-bit grey: EXIF orientation is applied (draw rects on the upright image), alpha (also PNG tRNS transparency) is flattened onto white, 16-bit and float images are scaled rather than clipped, CMYK images and colour images with an RGB/CMYK ICC profile are converted to sRGB; a grey page's profile (e.g. dot gain) is dropped and its pixel values kept. Unreadable EXIF is ignored. Modes it can't convert (LAB, HSV, signed 32-bit) are rejected with 422. Files over 64 MB are rejected with 413. What was changed is listed in the job's input_notes. A grey page gives a grey result (a 1-channel L PNG) unless the edit clearly added colour. That includes a grey page sent as RGB: an RGB upload whose channels never differ by more than 4 levels is stored as L (input_notes: grey_rgb_stored_as: "L"). So does a page with a faint colour cast within those 4 levels (a slightly warm or cool scan): the cast is dropped and the result is grey. A page with any pixel more colourful than that stays RGB.

Prompt composition (edit mode)

The server turns prompt and the rect labels into the instruction the model gets. The result is in run.prompt (and, with keep_debug, in debug_prompt*.txt).

• annotate: each labelled box gets its own sentence in list order (= colour order): In the red box region: remove the text. Unlabelled boxes share one sentence that carries the global prompt, placed where the first of them is: In the blue and green box regions: clean it. When every box is labelled, a global prompt follows as a whole-image sentence, unless it repeats one of the labels (case and trailing punctuation ignored). The prompt ends with Keep everything outside the marked regions exactly unchanged. The colored annotation lines must not appear in the output image. With a label on every box and no global prompt (the manga case) the prompt is byte-identical to the one before the 2026-09 review.
• mask and paint: one mask (or one painted colour) cannot tie a label to a region, so there is one instruction for all regions. The global prompt leads and the labels follow once each: In the masked regions of the first image: clean it. Region details: remove the text; hair. Without a global prompt the labels are the instruction. Paint mode says The flat white areas painted over the image mark the regions to edit. In the painted areas: ...; it sends no mask.
• Punctuation: trailing . , ; : … ‥ and their fullwidth/halfwidth forms are replaced by one full stop. A final ! ? 。 ！ ？ ． ｡ is kept. A label or prompt made only of punctuation is used as typed.

Do not rewrite prompts for rect-masked edit jobs (see "Prompt rewriting").

Job record

Besides status, progress, outputs and timings, a job carries the fields below. timings is kept for every outcome and summed over attempts (a re-queued job runs again): inference_s and total_s are totals, attempts counts the runs, the encode fields describe the last one. For a job ended by a server crash, total_s includes an estimate of the crashed attempt (to its record's last write; total_s_estimated: true).

• run: what actually ran, kept for done, failed and canceled jobs: seed and seed_source (request or random), outputs (per file its replay_seed and, except crop mode, the backend_seed the sampler got), render_size, the composed prompt (with the box sentences), and in crop mode windows (crop rect, render size, seed and prompt per window). With the paste-back's tone match, each output (crop mode: each window) also has tone_gain, tone_paper_px and tone_fit (ring_px, footprint_px, ...). For cache keys: build (the server's build id at the job's last run; each output also has its own build, since a resumed job can span builds), models (as in /healthz), backend (ComfyUI version and commit, torch and Python versions, ComfyUI's packages such as comfy-kitchen, the launch flags, the CUDA runtime and cuDNN, the backend's PYTORCH_CUDA_ALLOC_CONF, its custom nodes, the GPU and driver; read fresh for every job, null if the backend didn't answer) and loras (name, strength, file, sha256, size).
• debug_files (with keep_debug): the annotated/mask image and the final prompt as debug_prompt*.txt, served by GET /v1/jobs/{id}/inputs/{name}.
• error, error_code, retryable, max_retries (status "error"): see "Errors and retries".
• partial: true when outputs holds the finished images of a failed or canceled job (below).
• queue_position: 0-based, queued jobs only.
• backend_prompt_id: the ComfyUI prompt currently running for this job.
• backend_losses, presubmit_losses, oom_retries, requeues, note, waiting_since (while queued for a down backend), waited_for_backend_s: see "Backend outages and restarts".

Partial outputs. A job that ends "error" or "canceled" keeps the images it had finished: outputs lists them in order (so run.outputs[i] describes outputs[i]), the job has "partial": true, and GET /result?index=n serves them. Non-empty outputs no longer means success: check status == "done". Render the missing images by their replay seeds (seed S + i, num_images: 1). A queued job canceled after an earlier run (re-queued after a backend loss or a graceful restart) keeps what that run finished; a canceled queued job has progress: null.

Only finished images count. The comfy engine saves each image when its backend call completes. Crop mode has none (its one output is the stitched page). The diffusers engine renders all candidates in one call, so a failed or canceled job there normally has none.

Errors and retries

A failed job keeps its free-text error ("<Type>: <message>", as before) and carries error_code, retryable and max_retries (app/errors.py). retryable: true means the failure was not caused by the request, so the same request may succeed later. max_retries is how often to resend the unchanged request at most: 0 when not retryable, 1 when the job may be what crashed the server, null when the server sets no limit (use your own backoff). The list may grow: for an unknown code, go by retryable. A cancel is never an error (status: "canceled", no code). Jobs that failed before the codes existed get a code derived from their error text, in GET /v1/jobs/{id} and GET /v1/jobs.

error_code	retryable	max_retries	cause	client action
invalid_request	no	0	the request cannot run as sent, which showed only at run time: the backend rejected the graph (e.g. unknown sampler or scheduler), a LoRA was removed after the submit, a mode this backend lacks	fix the request
out_of_memory	no	0	the GPU ran out of memory. On the comfy backend the server has already retried once (resuming after the finished images)	make the job smaller (size, num_images, refs, cfg) or free the GPU; download the partial outputs and render the rest by replay seed
run_timeout	no	0	the prompt ran longer than its run timeout (the limit is in error)	make it smaller or cheaper (steps, size, sampler, cfg)
queue_timeout	yes	null	the prompt waited longer than the queue timeout (600 s) in ComfyUI's own queue	resend later
backend_lost	no	0	the backend went away twice while running this job; the job may be what crashes it	don't resend unchanged; report
backend_unavailable	yes	null	the backend could not be reached (the worker normally re-queues and waits instead)	resend later
backend_interrupted	yes	null	the backend prompt was interrupted by something other than this server	resend
backend_execution_error	no	0	a backend node failed while running the graph (not out of memory)	read error; change the request or report
backend_error	yes	null	any other backend failure: an HTTP error, a missing or unreadable answer	resend later
rewriter_error	yes	null	the prompt rewriter failed. If it gave no usable answer for a request with a fixed seed: retryable false, max_retries 0 (the same seed samples the same answer)	resend; with a fixed seed, change the seed
server_config	no	0	a setup problem: a missing model file, a backend older than 0.37, the LoRA folder not linked into the backend	the operator must fix it
server_stopped	yes	1	the server stopped (crash, kill, reboot) while the job ran	resend once at most; download partial outputs
upload_interrupted	yes	null	the server stopped while the job's inputs were being saved	resend
internal_error	no	0	anything else, e.g. a server bug	report; error has the details

Backend outages and restarts

If ComfyUI is unreachable, or restarts and forgets a prompt, the job goes back to the head of the queue (status: "queued", progress phase "waiting for backend") and the worker waits for the backend (retrying every 2 s, then every 4–5 s; /healthz shows "waiting_for_backend": true, waiting_since and running_job: null, and the job carries waiting_since; when it runs again the wait is added to waited_for_backend_s). The queue is not failed and the wait has no limit: unattended clients keep their own deadline. Rewrite jobs don't need ComfyUI: once one is next in line (or the waiting job is cancelled), the worker stops waiting and runs it. A job whose prompt had already reached the backend when it went away counts a backend_loss. A loss before submit (an upload or /prompt failed while /system_stats answers) counts a presubmit_loss: the job backs off 2, 4, 8 ... 30 s and fails as a retryable backend_unavailable on its 8th in a row. On the 2nd submitted loss the job fails with backend_lost, because it may be what crashes the backend.

Out of memory. A CUDA out-of-memory error from ComfyUI (reported as a node's execution error) re-queues the job once at the head of the queue (phase "retrying after out of memory", oom_retries: 1). If it runs out of memory again, it fails with out_of_memory and keeps its finished images as partial outputs. Nothing else is retried automatically.

Re-queued jobs resume. A job that goes back to the queue (backend loss, the out-of-memory retry, a graceful server stop; counted in requeues) keeps its seed, including one the server drew, and the images it had finished; the comfy engine renders only the rest (run.resumed_after = the number kept). Each output keeps its replay seed.

Every ComfyUI prompt is cancelled by id when the server stops waiting for it (timeout, error, job cancel). A job cancel is honoured within about 10 s even when the backend accepts connections but doesn't answer: polls use 3 s timeouts after a cancel, and a prompt whose cancel the backend didn't confirm is re-cancelled in the background every 15 s for 10 minutes. A graceful server stop (systemctl --user restart qwen-image-server) puts the running job back in the queue for the next start (note) and cancels its prompt; the job then resumes (see "Re-queued jobs resume"). After a crash the next start marks the job as an error (server_stopped, retry once at most), keeps its finished images as partial outputs and cancels the prompt it left behind. A cancel that arrives while the engine fails is recorded as "canceled", not as an error.

Busy vs down in /healthz. The server asks ComfyUI's /system_stats (1 s to connect, 3 s to answer). A refused connection or no connection is down: backend_up: false, status: "degraded". A connection without an answer while a job runs is busy, not down (a running prompt can hold up ComfyUI's event loop): backend_up: true, backend_busy: true, and the last VRAM figures with their age in vram_age_s. The same silence while idle stays "degraded", since a hung backend looks exactly like that.

Queue counts. A job canceled while queued leaves the queue at once, so queue_length and the other jobs' queue_position are right immediately.

Timeouts and polling

These apply to the comfy engine; the diffusers engine has no timeouts.

• Queue timeout: a prompt may wait QIS_COMFY_QUEUE_TIMEOUT (600 s) in ComfyUI's own queue.

• Run timeout: the clock starts when ComfyUI starts executing the prompt. The limit scales with the prompt's work:

  timeout = max(floor, min(ceiling,
            2 × (60 s + steps × 1.15 s × MP × evals × (2 if cfg > 1) × (1 + 0.1 per LoRA))))
  

  MP is the DiT input in megapixels: the output plus every condition image (an edit is at least twice its output). evals is the sampler's model evaluations per step: 2 for heun, dpm_2, dpm_2_ancestral, dpmpp_2s_ancestral(_cfg_pp), dpmpp_sde(_gpu), exp_heun_2_x0(_sde), seeds_2 and sa_solver_pece; 3 for heunpp2 and seeds_3; 1 for all others. dpm_adaptive picks its own step count and always gets the ceiling. The floor is QIS_COMFY_RUN_TIMEOUT (900 s) and the ceiling QIS_COMFY_RUN_TIMEOUT_MAX (3600 s); a floor set above the ceiling wins. Note that QIS_COMFY_RUN_TIMEOUT is no longer a cap: to cap the run time, set the ceiling. Examples: a 1 MP manga window at 25 steps keeps 900 s; a 2048² edit (8.4 MP) at 25 steps and cfg 4 gets 1085 s; a 2048² generate at 100 steps, cfg 4 and 4 LoRAs 2821 s. A prompt whose limit is above the floor logs it at info level. The constants and their fit to the job history are in app/comfy_engine.py.

• Polling: the server polls ComfyUI every 0.2 s, so it notices a finished image, or passes on a cancel, within 0.2 s. progress is written to the job record when it changes, or at most once a second when it doesn't. A failed poll is retried every 1 s; 5 in a row (about 5 s) and the backend counts as gone. A 200 whose body is not a JSON object (a proxy's error page) is a failed poll too; 5 of those in a row fail the job as a retryable backend_error (something answers, so the backend is not counted as lost).

• Clients: poll GET /v1/jobs/{id} about once a second. A queued job waits without limit while the backend is down, so an unattended client needs its own overall deadline.

Limits

• Canvas. Generate: see width/height (at most 4.19 MP). Edits keep the page's aspect ratio within output_resolution² and are never upscaled: a 118 × 244 screenshot renders at 96 × 224 (sides floor to a multiple of 32). An edit output of exactly 1024 × 1024 is nudged to 992 × 992. In crop mode output_resolution is ignored: each window renders at its own size, floored to 32, with no cap.
• Cost. Above edits of about 3 MP the cost per step grows faster than the pixel count, because the 12 GB card starts offloading. Per step and MP of DiT input (output + condition images) the job history gives about 0.28 s for ~1 MP edits, 0.49 s for 3 MP edits and 1.07–1.13 s for 4.1–4.2 MP edits. A 4.1–4.2 MP edit took 250–270 s at 25 steps (490 s with cfg 4 and a LoRA). The run-time fit covers up to about 8.4 MP of DiT input (a 2048² edit) with euler and er_sde only; above that (a 2048 edit with several refs reaches about 12 MP, crop windows are uncapped) it is extrapolated, and the 1 h ceiling bounds it.
• Rects. annotate takes at most 8 rects, one per colour, in list order: red, blue, green, yellow, purple, orange, cyan, magenta. Boxes named 5th to 8th were skipped in about 1 trial in 4 in the 2026-09 review, which is why examples/manga_clean.py sends at most 3 boxes per call (2 with floating text). The annotation stroke (about 3 px at 1024) is drawn inside the rect.
• Seeds. Output i of seed S is rendered exactly like output 0 of seed S + i, so a job with seed S + 1 and num_images: 3 shares 2 of its 3 outputs with seed S.
• Encode cache. A rewrite job (it asks ComfyUI to free its memory) or a backend restart empties ComfyUI's cache, so the next regenerate encodes again (see "Encoding reuse").
• Queue. One GPU worker and one FIFO queue with no priority: keep 1–2 jobs in flight per client, or one client's batch starves the others.
• Other bounds: steps 1–100, num_images 1–4 (1 in crop mode), up to 9 refs, up to 4 LoRAs, 64 MB per uploaded file.

Prompt rewriting (mode: "rewrite")

Text-to-image prompt (no images -> PE-T2I):

curl -s http://localhost:8500/v1/jobs \
  -F 'payload={"mode":"rewrite","prompt":"Tokyo alley at rainy night, stray dog, Daido Moriyama style"}'
# result: {"rewritten_prompt": "<~2000-char English prompt>", "wh_ratio": "2:3", "variant": "t2i"}

Edit instruction (with images -> PE-I2I, the vision rewriter):

curl -s http://localhost:8500/v1/jobs \
  -F 'payload={"mode":"rewrite","prompt":"make it look like an old photo"}' \
  -F image=@photo.png            # becomes <image1>; extra -F ref=@... are <image2>...
# result: {"rewritten_prompt": "...", "wh_ratio": "", "ratio_follow": "<image1>", "variant": "i2i"}

ratio_follow names the input image whose aspect ratio the edited output should keep; wh_ratio is set instead when the rewriter picks a new canvas. Feed rewritten_prompt into a follow-up mode: "edit" job with the same images in the same order (the prompt refers to them as <image1>...).

Both run the official Qwen-Image-2.1 rewriters (pottokao Heretic GGUF builds, Qwen Research License) via llama.cpp as normal queued jobs: the ComfyUI backend frees VRAM, llama-server takes the GPU, the next image job reloads. Official message format and sampling (images first, each capped at 1 MP; temp 1.0 / top_p 0.95 / top_k 20); the I2I model reasons in a <think> block that the server strips. Any input language; seed makes it reproducible. Falls back to CPU (minutes) when CUDA is unavailable. Requires QIS_BACKEND=comfy. Advertised as prompt_rewrite in /healthz. Setup (the llama.cpp b11120 build and the GGUF downloads, with file names): deploy/README.md item 4.

Do not rewrite prompts for rect-masked edit jobs — the official mask examples bypass the rewriter (prompt_extend=false); it destroys the box-color/mask references the edit relies on.

LoRAs (loras, comfy backend)

Put Qwen-Image-2.1 LoRA files (.safetensors, ComfyUI key naming — e.g. ai-toolkit output) into loras/. They are listed immediately, no restart:

curl -s http://localhost:8500/v1/loras
# {"dir": ".../loras", "loras": [{"name": "qwen_unlock", "title": "...", "rank": 32,
#   "trigger_words": [...], "compatible": true, "info": {...sidecar...}}]}

Use them per job by name (file stem), up to 4. Strength defaults to the range [-2, 2] (guards against typos like 8 for 0.8); slider-style LoRAs made for larger values declare their own range in the sidecar, e.g. {"strength_range": [-20, 20]} (max ±100). GET /v1/loras reports each LoRA's effective strength_range for client sliders.

-F 'payload={"mode":"generate","prompt":"...","loras":[{"name":"qwen_unlock","strength":0.8}],
             "sampler":"er_sde","scheduler":"beta","true_cfg_scale":4,"steps":25}'

LoRAs apply to the diffusion model only (the text encoder is stock Qwen3-VL) and work in generate and edit jobs. Optional sampler / scheduler take any ComfyUI KSampler name (default euler / simple). On the comfy backend true_cfg_scale > 1 now applies even without a negative prompt (empty uncond, as LoRA authors usually assume); CFG doubles the sampling cost.

An optional sidecar loras/<name>.json is returned as info — e.g. the author's recommended settings; the server never applies them itself. One-time backend setup (already done on this machine; deploy/README.md item 3):

ln -s ~/qwen-image-server/loras ~/comfyui-bench/models/loras/qis

examples/manga_clean.py is the fully commented reference client for manga cleaning with client-side windows (every payload field explained, runs out of the box against the test page); it is single-pass, without the lab's candidates, judging and retries. scripts/client_example.py is a thinner general-purpose client.

Encoding reuse (regenerate)

Advertised in /healthz features as "reuse_encoding" and "encode_cache_report".

Encoding the prompt together with the condition images is a large, fixed share of an edit (~10 s of a 13 s 1 MP edit with two refs on the comfy backend). A regenerate that only changes the seed doesn't need it again:

• comfy backend: condition images are uploaded to ComfyUI under content-addressed names (qis_<sha256 of the PNG, 24 hex>.png, PNG encoded deterministically). Identical inputs give an identical graph, so ComfyUI's node cache serves the image loads and the text/image encode and only sampling + decode run. reuse_encoding: false uploads under unique names instead, which forces a fresh encode. A text-only generate has no images to rename, so ComfyUI may still serve its prompt encode from cache (encode_cached says so).
• diffusers backend: only text-only generates use the disk embed cache; reuse_encoding: false skips the cache lookup (the fresh encode is still written back). Edits are always encoded.

Finished jobs report timings.encode_cached (true = the encode was served from a cache). On the comfy backend a job can make several backend calls (one per image with num_images > 1, one per crop in region_mode=crop); encode_cached is true only if every call hit, and timings.encode_cached_calls gives the count as "hits/calls", e.g. "1/3".

A regenerate can only hit if ComfyUI still holds the encode from the earlier prompt. Launch the backend with --cache-lru <N> (deployed: 256). N counts cached node results, not prompts. Inputs that stay the same across jobs (the model loaders, a constant LoRA or ref) are one entry in total; a fresh window adds 5 entries, +1 per condition image that changes per window (e.g. the mask in mask mode), +3 per extra num_images candidate; a seed-only retry adds 3. So a retry still hits after about (N - 3) / (entries per window) other fresh windows: about 50 at 256 for one-image windows, about 22 for three candidates. The classic cache (--cache-classic) and v0.37's default cache (no flag) keep only the current prompt, so only an immediate retry hits. A rewrite job and a backend restart empty the cache. encode_cached_calls after a regenerate shows which case you are in.

How masking works

Qwen-Image-2.1 has no mask tensor input; local editing is conveyed through the condition images (officially: colored circles/boxes, painted regions, or a separate mask image) and the prompt. The model re-renders the whole canvas, so the server composites the original image back outside the rects afterwards (composite_back): beyond the feathered footprint the output is the input, byte for byte. Inside the rects it is new pixels, and the 16x VAE redraws fine screentone rather than preserving it. region_mode=crop additionally restricts the model's view to a context window around each rect (merged when they overlap), which avoids resolution loss on large pages.

The model's paper comes back a few levels darker than the scan (median 4 levels in the 2026-09 review), which leaves grey patches in white balloons. With tone_match (default on) the paste-back fits a per-level luma correction on paired pixels whose mean level (scan + render) / 2 is >= 200 (paper and light tone), of two kinds: in a ring just outside the feathered footprint, reaching 16 px past rect + expand + 3 × feather, where the scan is within 24 levels of the render; and inside the footprint, where it is within 12 levels (the balloon's own paper around the erased text, which makes the fit work when the ring is all art). Pairs are selected and binned on that mean level, so noise on either side biases no bin (a floor on the scan side alone lifted light tone near 200). The pairs are binned by mean level in steps of 8 levels; each bin with at least 50 pairs gets the ratio of its two medians (scan median / render median), clipped to ±3 %, and levels in between are interpolated, so white paper and light tone, which the model darkens by different amounts, get their own corrections. Each pixel's R, G and B are scaled by the same factor, so hues don't shift and a grey page stays grey. It changes only pixels the render is blended into. The fit is stored per output in run.outputs[i]: tone_gain (the correction at white), tone_paper_px, and tone_fit (ring_px, footprint_px, scan_median, render_median, and levels as [render median, ratio, pairs] per bin). It costs about 40 ms on a 4 MP page with 8 rects.

Limits: one curve for all rects of an image (rects on different paper share it), luma only, at most ±3 % per level, only render levels >= 200 (paper and light tone) are fitted and the correction fades to none over the 64 levels below the lowest fitted one, so ink and dark art are untouched, and no correction with fewer than 200 paper pixels (dark pages, tiny images). For per-region fits use client windows (examples/manga_clean.py fits one gain per call, per channel).