Use a reference image in Forge

First decide what must be preserved. Use Img2img for source pixels, ControlNet for structure, Reference-only or IP-Adapter for visual guidance, and the separate PhotoMaker V2 Space for a person’s identity.

Img2img starts diffusion from an encoded source image, so denoising governs how much of that canvas remains. IP-Adapter supplies image-prompt features alongside the text prompt without using the reference as the starting canvas.

A conventional ControlNet pair carries an explicit map such as pose, edges, or depth. IP-Adapter carries image features such as content or style. They can be combined, but they solve different jobs.

It is a built-in ControlNet preprocessor family that uses the reference during sampling and requires no separate control-model file. Current Original Forge exposes reference_only, reference_adain, and reference_adain+attn.

No separate ControlNet model is required by the inspected Reference preprocessors. They set do_not_need_model, so the integrated UI hides its Model control for this route.

It controls the balance used by the Reference preprocessor; the inspected default is 0.5. Treat it as a comparison variable, not a quality score, and change it only after one same-seed baseline.

Open ControlNet Integrated in Txt2img or Img2img, then choose the IP-Adapter Control Type. The current labels include CLIP-ViT-H (IPAdapter), CLIP-ViT-bigG (IPAdapter), and InsightFace+CLIP-H (IPAdapter).

The integrated scanner uses the effective models/ControlNet directory and any configured extra ControlNet path. Image-encoder downloads use models/ControlNetPreprocessor.

Use the preprocessor required by the exact adapter variant. The maintainer matrix maps standard SD 1.5 variants mainly to ViT-H, sd15_vit-G to ViT-bigG, and SDXL variants to either encoder depending on the filename. FaceID is a separate and less reliable boundary.

No general cross-family support is established. Use adapter weights published for the same base family as the active checkpoint.

This audit does not establish a universal FLUX reference-image route in Original Forge. Do not reuse SD 1.5 or SDXL files; follow an exact publisher contract and label the result unknown until it passes.

Current code logs that the CLIP image processor resizes and center-crops non-square images. A subject near an edge may be cropped away, so use a centered square crop for the diagnostic baseline.

Do not treat it as an exact identity guarantee. The IP-Adapter paper says the base method produces resemblance in content and style rather than the high subject consistency of personalization methods. Face-oriented variants add different dependencies and failure modes.

Yes in principle, and the IP-Adapter paper demonstrates image prompting with structural controls. In Forge, prove the IP-Adapter unit and the structural unit separately with the same base receipt before combining them.

Prove one Single Image first. The maintainer explicitly declined the old WebUI multi-input mixture and recommended separate ControlNet units for similar logic; an open report says batch input used only its first image. Test each unit alone, then combine them. Use PhotoMaker V2 when several photos of one person are the actual requirement.

For a dedicated identity workflow, use the separate PhotoMaker V2 Space with clear photos and the required img token. IP-Adapter FaceID exists in the code path, but open reports show unresolved variant and dependency problems, so it is not presented as the default reliable route.

At the inspected commit it appears under Spaces as PhotoMaker V2. Install and Launch open a separate local Gradio app; it is not a normal ControlNet preprocessor.

No. The inspected wrapper constructs its own SDXL pipeline with SG161222/RealVisXL_V4.0 and separate adapters, independently of the checkpoint selected in the main Forge header.

Use the img trigger exactly once, after the person class—for example, “a photo of a woman img”. The wrapper checks for a missing token and rejects multiple occurrences.

Start with two clear, varied views if available. The paper found the largest identity-fidelity improvement when moving from one image to two, with diminishing gains and a possible tradeoff in text alignment as inputs increase.

No. It generates a new portrait through a personalized SDXL pipeline. Identity similarity can vary, and neither the paper nor the wrapper promises pixel-exact face replacement.

The current Forge wrapper has an optional doodle control using an SDXL T2I Adapter, but its own interface warns that quality may decrease. Prove identity with doodle off before adding that condition.

Image features are not perfectly separable. Use a simpler crop, compare Reference-only with IP-Adapter, and add an explicit pose or depth ControlNet only after the appearance route passes.

Record Original Forge commit, checkpoint or Space pipeline, exact adapter and encoder filenames, source image crop, prompt, seed, dimensions, sampler, scheduler, steps, every unit, weight, range, and same-seed enabled/disabled outputs.