Lumera Guide 2026: Image to 3D and Image to Blender, Explained

You feed it one photograph, and it hands you back a scene you can open in Blender: separate object instances you can select and move, parametric lights whose color and intensity you can dial in an inspector, and a global HDR environment — not a baked mesh, not a video, an actual editable 3D scene. Among image to 3d research published in 2026, Lumera is the first to make lights a first-class, countable output. That is the promise of Lumera, the "engine-native editable 3D world reconstruction" research from Tsinghua University, Nankai University, UESTC, Sun Yat-sen University, Nanyang Technological University, and the industry team at Ophilus.AI, published as arXiv preprint 2607.20889 in July 2026.

This guide explains what Lumera is, how its image-to-3D pipeline works step by step, what the numbers in the paper actually say, how its image-to-Blender output path works, and — importantly — what Lumera cannot do yet. Everything below is sourced from the official Lumera paper (arxiv.org/html/2607.20889v1) and the official Lumera project page (haidilao0328.github.io/Lumera).

🎯 Key Takeaways (TL;DR)

• Lumera = Light-aware Unified Engine-native Reconstruction and Assembly: a benchmark plus a reference pipeline that parses one input image into an engine-native, editable 3D scene — object instances, meshes, parametric lights, and an HDR environment probe.
• It targets the gap most image-to-3D tools leave open: instead of a fused geometry proxy, Lumera outputs things you can inspect and move in standard engines like Blender and UE5 — oriented object boxes, per-object meshes, and countable lights.
• The Lumera-2K dataset is built from 2,513 UE5 projects with 3.73M components, 63M object instances, 102.6K engine-native parametric lights, and 95.1K camera views.
• Measured results: Lumera-Box leads DetAny3D, SpatialLM, N3D-VLM, and WildDet3D on merged mAP (0.1141), IoU-B (0.2472), and F-score (0.2762); Lumera-Light recovers 99.8% of non-empty light scenes but strict per-light localization remains hard (F1 0.209 at 0.5 m).
• The image to Blender path is explicit: a UE-to-Blender light adapter maps UE5-native units (lumen/candela, Kelvin flags, spotlight cones, attenuation) onto Blender Cycles' physical light fields, loss-preserving — so what Lumera reconstructs stays editable inside Blender.
• Reality check: code and the Lumera-2K dataset are marked "soon" on the official project page, the paper is a preprint under a CC BY-NC-SA 4.0 (non-commercial) license, and per-light localization is still the weakest link.

目录 (Table of Contents)

1. What is Lumera?
2. Why image-to-3d is harder than it looks
3. How Lumera works: the 4-stage pipeline
4. Image to Blender: the UE-to-Blender light adapter
5. The Lumera-2K dataset
6. Benchmarks: what the numbers say
7. Limitations you should know before waiting for the code
8. How to follow or try Lumera today
9. FAQ

What is Lumera?

Lumera stands for Light-aware Unified Engine-native Reconstruction and Assembly. It is two things at once:

1. A benchmark — Lumera-2K, a dataset of game-engine scenes with dense ground truth for objects, cameras, HDRI, and countable lights, built from 2,513 UE5 projects.
2. A reference pipeline — an end-to-end method that takes a single image (plus its recovered point cloud) and produces an assembled, editable, engine-native 3D scene. In other words, Lumera is not a single model but a full image to 3d workflow.

The Lumera research comes from a team spanning Tsinghua University, Nankai University, the University of Electronic Science and Technology of China, Sun Yat-sen University, Nanyang Technological University, and Ophilus.AI, with Hao Zhao (Tsinghua) as corresponding author. It was released as preprint arXiv:2607.20889 on July 23, 2026.

The key phrase in Lumera's positioning is engine-native. The demo scenes on the official page — a graveyard, a burger shop, a cottage house, a lunar base, an industrial site, a Japanese castle, a haunted mansion, and a top-down interior — are not just point clouds or renders: Lumera turns a single image into object boxes, parametric lights, and an HDR probe that are "loadable in Blender / UE5."

💡 If you are searching for "image to 3d" tools: most consumer tools (Meshy, Tripo, TRELLIS-style generators) reconstruct a single object. Lumera reconstructs a scene — multiple instances with layout, plus the lighting rig — which is a different and harder problem.

Why image-to-3d is harder than it looks

The paper's introduction states the problem plainly: turning a single image into an editable 3D scene matters for game production, virtual production, simulation, robotics, and asset creation — and it has recently become a prerequisite for controllable video generation, where video diffusion models condition on explicit 3D scenes for camera control.

But whether the downstream tool is Blender, UE5, or a 3D-conditioned video model, the requirement is the same, and existing single-image methods fall short in three predictable ways (quoted categories from the paper):

• Room-scale geometry only — systems like SpatialLM-class parsers stop at indoor layouts and don't scale to the game-scale scenes Lumera targets, with dense foreground assets and mixed indoor/outdoor structure.
• Baked or global illumination — reconstruction methods capture lighting as an uneditable afterthought (a baked texture or one global environment), so you cannot re-light the scene.
• Text-driven generation — promptable 3D generators invent plausible scenes but don't preserve the input image's actual content and layout.

What users actually need, per the paper: "object instances that can be moved, meshes that can be replaced or refined, and lights whose positions, colors, and intensities can be inspected and edited." That is the checklist Lumera optimizes for.

How Lumera works: the 4-stage pipeline

The Lumera pipeline runs four stages — structured perception first, constrained editing second:

A few details worth understanding, because they explain how Lumera behaves and why the benchmarks read the way they do:

• Depth Anything 3 is the metric bridge. It provides the camera, depth, and colored point cloud; Lumera-Box and Lumera-Light operate on that structured data, not raw pixels.
• The 3D box is the instance identity. When SAM-guided masks and box inliers disagree (occlusion, calibration noise), the box wins — masks only improve the RGB crop fed to SAM3D, which reconstructs each object's textured mesh in normalized coordinates before it is transformed back into the scene.
• Shells are handled separately. Walls, floors, ceilings, and terrain don't go through per-object mesh recovery; a dedicated shell pipeline covers the large unbounded surfaces.
• Refinement is bounded on purpose. The Lumera Generator agent can only edit fields in the current stage's scope — object yaw/scale during geometry; existing lights, environment strength, and exposure during lighting. A structured Verifier report with a termination bit decides when to stop, and every executed edit is validated so bad edits can be rolled back.

⚠️ Note the pattern: Lumera is "VLM-adapted parsing + constrained assembly," not one giant generative model — every Lumera component is a measurable unit with its own benchmark. Each component is a measurable unit with its own benchmark — which is exactly what makes the paper's results auditable.

Image to Blender: the UE-to-Blender light adapter

This is the part that matters for anyone searching "image to blender." Lumera's ground truth lives in Unreal Engine 5 units; Blender's Cycles renderer has different, physically-based light fields. The paper's appendix H.3 specifies a UE-to-Blender light adapter that Bootstrap runs before writing any scene to Blender:

• Identity and world pose preserved — every light keeps its name and position; the adapter never creates lights that aren't in the parsed set.
• Units mapped to physical quantities — UE5 lumen/candela values are translated to Cycles' radiometric equivalents.
• Color handled two ways — taken directly from RGB, or converted from Kelvin when the source light carries a temperature flag.
• Cone and attenuation written through — UE5 SpotLight cone angles and attenuation radius land in the corresponding Blender fields.

The adapter is described as loss-preserving, which keeps the Lumera agent's lighting search space consistent between engines. Combined with the IntrinsicHDR environment probe, this is what makes the Lumera output a genuinely editable image to Blender deliverable: after reconstruction you can open the assembled scene, select any light, and re-light the shot inside Blender.

✅ Practical reading: if your goal is "image to Blender with editable lights," Lumera is currently a research preview of that workflow — the paper demonstrates the full image to blender flow (the project page shows input image → generated Blender scene → re-rendered results for a graveyard and a Japanese interior), but Lumera's code is not public yet at the time of writing.

The Lumera-2K dataset

Lumera-2K is the benchmark half of the Lumera release, and its scale is the paper's quiet flex:

Two things make Lumera-2K unusual for the image to 3d field:

1. Scene-aware camera planning. Rather than random screenshots, the authors use a five-stage camera planner (foreground filtering and XY clustering, indoor room segmentation with multi-mode candidate sampling, occlusion-aware visibility filtering, greedy coverage with diversity constraints, and image-level QA filtering) rendered headlessly in UE5 — so every image is guaranteed to actually contain content Lumera can recover.
2. Countable, engine-native light labels. Every light annotation carries UE5-native fields — lumen/candela units, a Kelvin-temperature flag, spotlight cone angles, attenuation radius — which is what makes "parametric lights" a measurable reconstruction target instead of a vibe.

Benchmarks: what the numbers say

On a sanitized box benchmark drawn from the merged Lumera-2K val+test split, Lumera-Box was compared against four strong 3D detection baselines: DetAny3D, SpatialLM, N3D-VLM, and WildDet3D:

⚠️ Honest interpretation: these are hard, unforgiving metrics — 0.11 mAP is "best" in a field where nobody is close to saturated. The honest takeaway is that Lumera defines the first measurable leaderboard for editable-scene parsing (objects and lights), not that image to 3d scene reconstruction is solved. Within that new yardstick, Lumera-Box and Lumera-Light set the bar others now have to clear.

Limitations you should know before waiting for the code

The paper's own limitations section (Section I) is unusually candid, and the project page adds two more practical caveats:

• Code and dataset are "soon." The official Lumera project page marks both "Code · soon" and "Lumera-2K · soon" — as of this writing you can read the paper and study the demos, but you cannot run Lumera yourself yet.
• Non-commercial license. The preprint is distributed under CC BY-NC-SA 4.0, so plan for research/educational use, not production pipelines, until the authors say otherwise.
• Box parsing is strongest overall, but not saturated — detection quality has a long way to go.
• Orientation and relation recovery remain weak points — object yaw errors and inter-object relations are the hardest parts of layout.
• Light localization is the bottleneck — the pipeline nearly always knows that a scene has lights (0.998 recall) but struggles to pin down each light (F1 0.209 at 0.5 m), and light carriers/intensity remain incomplete.
• Large unbounded scenes still drift geometrically, and output validity sometimes needs model-level constraints.
• Engine generalization is narrow — the demonstrated Lumera path is UE5-native data with a UE-to-Blender adapter; other engines are future work.

How to follow or try Lumera today

1. Read the Lumera paper: arxiv.org/abs/2607.20889 (HTML version: arxiv.org/html/2607.20889v1).
2. Study the official Lumera project page: haidilao0328.github.io/Lumera — it includes an interactive viewer where you can toggle the input image against the recovered object boxes and parametric lights, plus eight demo scenes (Graveyard, Burger Shop, Cottage House, Top-Down Interior, Lunar Base, Industrial Site, Japanese Castle, Haunted Mansion) with their full geometry and lighting iteration histories.
3. Watch for the code release: both the code and the Lumera-2K dataset are labeled "soon" — check the project page or the arXiv page for updates.
4. Need an image to 3d deliverable today? For single objects, consumer generators (Meshy, Tripo, TRELLIS-based demos) ship now; for whole scenes with editable lighting, Lumera-style engine-native reconstruction is the direction to watch. For manual work, the classic path — photogrammetry or hand-modeling plus re-lighting in Blender — still gives the most control.

💡 Citation: if you reference Lumera in academic work, the project page provides a ready BibTeX entry (chen2026lumera, Preprint, 2026).

🤔 FAQ

Q: What exactly is Lumera?

A: Lumera (Light-aware Unified Engine-native Reconstruction and Assembly) is a 2026 research benchmark and reference pipeline that reconstructs an editable, engine-native 3D scene — object instances, per-object meshes, parametric lights, and an HDR environment — from a single image. Think of it as image to 3d taken to scene scale, with image to Blender as the delivery step.

Q: Is Lumera available as a tool I can use right now?

A: Not yet. As of August 2026 the paper and project page are public, but code and the Lumera-2K dataset are both marked "soon" on the official page. There is no hosted demo or app at this time.

Q: How is Lumera different from other image-to-3d tools?

A: Most image to 3d tools reconstruct a single object or a fused geometry proxy. Lumera reconstructs whole scenes into engine-native structures — oriented boxes with open-vocabulary labels, individual meshes, countable parametric lights (x, y, z, r, g, b, I), and an HDR probe — so the result can be edited in Blender or UE5 rather than merely viewed.

Q: What does "image to blender" mean in the context of Lumera?

A: In Lumera, image to blender is the delivery step: the pipeline assembles the reconstructed scene and writes it into Blender with a loss-preserving UE-to-Blender light adapter: UE5 lumen/candela units map to Cycles' physical light fields, Kelvin-temperature flags convert to colors, and spotlight cone/attenuation parameters carry over — so you can re-light the reconstructed scene inside Blender.

Q: What data was Lumera trained and evaluated on?

A: Lumera-2K, the Lumera project's dataset, built from 2,513 UE5 projects containing 3.73M components, 63M object instances, 102.6K engine-native parametric lights, and 95.1K camera views, with scene-aware camera planning and countable light annotations.

Q: How well does Lumera perform?

A: On the sanitized Lumera-2K box benchmark Lumera leads DetAny3D, SpatialLM, N3D-VLM, and WildDet3D on merged mAP (0.1141), IoU-B (0.2472), and F-score (0.2762), though WildDet3D stays ahead on anchor recall. For lighting, Lumera-Light achieves 0.998 non-empty-scene recall, a 0.261 m median position error for matched lights, but only 0.209 F1 at strict 0.5 m localization.

Q: Can I use Lumera commercially?

A: Not on the current license. The preprint is CC BY-NC-SA 4.0 (non-commercial, share-alike), and there is no commercial product yet — watch the project page for any change.

Q: Who is behind Lumera?

A: Lumera is a collaboration between Tsinghua University, Nankai University, the University of Electronic Science and Technology of China, Sun Yat-sen University, Nanyang Technological University, and the company Ophilus.AI; the corresponding author is Hao Zhao (Tsinghua).

总结与建议 (Summary)

Lumera reframes the image to 3d problem around what 3D artists actually need: not pixels, not a blob — but named object instances, replaceable meshes, countable lights, and an HDR environment that survive the trip into Blender and UE5. The Lumera-2K benchmark turns "editable scene reconstruction" into measurable numbers for the first time, and Lumera's UE-to-Blender light adapter shows the image to blender workflow end to end, from a single photo to a re-lightable scene.

The caveats are equally clear: Lumera is a preprint, code and data are "soon," the license is non-commercial, and per-light localization has a long way to go. If you work in game production, virtual production, or 3D-conditioned video generation, put Lumera on your watchlist — and if you need results today, combine a single-object image to 3d generator with manual re-lighting in Blender while the scene-level version of this dream matures.

Resources:

• Paper: arxiv.org/abs/2607.20889
• Project page: haidilao0328.github.io/Lumera
• Underlying components referenced in the paper: SpatialLM, Depth Anything 3, SAM3D, IntrinsicHDR, VIGA