Image-to-3D Without the Clay-Like Look: Geometry, Materials, and Asset Validation

01First locate the failure: geometry, shading, or delivery

A common first mistake is to respond to a blurry surface by simultaneously increasing polygon count, sharpening textures, and lowering roughness. The geometric normal is determined by the actual surface, whereas the shading normal can be modified through vertex interpolation or normal mapping. The latter changes highlights, but cannot reopen a filled hole or restore a missing silhouette. PBRT’s distinction between these two normals provides a practical diagnostic starting point: establish whether the shape exists before checking whether it reflects light correctly.[1]

Keep the camera, exposure, and color management fixed, and save four diagnostic outputs of the same asset. The first shows only silhouette and depth. The second uses a uniform neutral material, disables normal maps, AO, and cosmetic post-processing, and compares face normals with smoothed normals. The third displays unlit base color. The fourth restores the complete PBR material: rotate the light with the camera fixed, then rotate the camera with the light fixed. This is the diagnostic sequence proposed in this report; each output answers a different question.

If silhouette and depth are already incorrect, prioritize input constraints, geometric representation, and surface extraction. If the structure is correct with face normals but edges melt under smooth shading, prioritize split normals, smoothing groups, and the tangent basis. If the unlit color retains directional shadows or fixed highlights, prioritize material–lighting separation. If the asset looks correct in the authoring application but clay-like on the web or in another engine, check export and import settings before replacing the generation model. Our existing report From Visual Representations to Editable 3D Assets discusses representation conversion; this report further traces local information loss along that conversion chain.

• Silhouette and depth: verify that the shape actually exists.
• Uniform material and normal comparisons: separate geometric errors from shading errors.
• Unlit color: check whether shadows and highlights are baked into the color.
• Light rotation, camera rotation, and reproduction in the target engine: check material behavior and delivery consistency.

02A single image is underconstrained: more pixels are not more observations

Image formation can be written as I = R(G, M, L, C): image I is jointly produced by geometry G, material M, lighting L, and camera C. In a single image, a dark region may be a groove, paint, or a shadow; a bright edge may arise from a bevel or simply a specular reflection. Multiple parameter combinations can explain similar pixels. Minimizing RGB error from the input viewpoint alone can therefore make texture explain features that should belong to geometry. The joint optimization of shape, materials, and lighting in nvdiffrec illustrates why these variables should be analyzed together, rather than treating photographic colors directly as surface properties.[2]

For a real product, additional side, rear, and close-up views should show the same object in the same assembly state, with camera information or a scale reference retained. For concept artwork, generated views can suggest candidate shapes, but do not become independent evidence about a physical object. Generating eight sharp images from one blurry reference may simply complete the same error in eight apparently consistent ways.

Inspect the input at the level of critical parts: has foreground extraction removed a thin rod; has a dark opening been treated as background and filled; have reflections made adjacent parts appear fused; and how many pixels still span a groove after resizing? Super-resolution can produce an appearance that better matches learned priors, but cannot prove recovery of a groove depth never recorded in the source image. Where vents, threads, connectors, or mating surfaces require precision, supplement the image with drawings, photographs, or measurements, and use constrained local modeling where necessary.

03From pixels to voxels: establish a scale budget for detail

Under an approximate pinhole camera model, with a feature near the optical axis and its width direction parallel to the image plane, a physical width w projects to approximately p = f_px × w / z, where f_px is focal length in pixels and z is camera-space depth. Off-axis position, tilt, and occlusion change the effective projection; the following discussion uses only an interpretable first-order approximation. If p is already below one pixel, a single image may preserve the feature’s contribution to brightness, but will struggle to resolve its two edges reliably, let alone guarantee recovery of its cross-section.

For a regular output grid or sparse voxel representation, Δ ≈ B / R estimates spatial sampling pitch: B is the side length of the bounding extent before normalization, and R is the resolution along that axis. The number of cells spanning a feature, k = w / Δ, helps identify risk, but is not a certificate of quality. Grid orientation, feature phase, the encoder, the decoder, and the topology expressible within each cell all affect what survives. Nor can different algorithms be compared using R alone.

Consider a constructed example. A product has a bounding extent of 300 mm and an output resolution of 1024, giving Δ ≈ 0.293 mm; a 0.5 mm rib spans only about 1.71 cells. With f_px = 1000 and a camera depth z = 600 mm, it occupies only about 0.83 pixels in the input image. Raising output resolution to 1536 increases the span to 2.56 cells, but the input evidence remains 0.83 pixels. Additional computation may improve prior-driven completion; it does not establish that the physical rib has been reconstructed accurately.

Recalculate these scales after the cropping, background removal, and resizing actually used by the model. A 4K source file does not mean the network sees 4K local evidence; a million output triangles do not imply a million independent pieces of structural information. Latent token count, VAE downsampling factor, voxel resolution, and final polygon count describe different stages and cannot substitute for one another. Prioritize details that affect silhouette, occlusion, functional recognition, or subsequent editing.

• Input budget: how many pixels cover a critical structure in the actual model input?
• Representation budget: how do feature width, wall thickness, and spacing between neighboring surfaces compare with the sampling scale?
• Delivery budget: does the structure survive remeshing, decimation, and baking?

04Representation choice: do not blame the clay-like look on SDFs alone

A signed distance field, or SDF, describes distance to a surface together with an inside/outside sign. Closed solids can contain cavities, finite-thickness thin shells, and multiple boundaries; the claim that “SDFs can only generate solid objects” is inaccurate. Open zero-thickness surfaces, non-manifold junctions, and multiple closely spaced surfaces pose different representational challenges. In practice, distinguish the representation itself from data processing: making all training assets watertight, closing openings, or deleting internal parts loses information before the model starts learning.

The public shape-generation pipeline in Hunyuan3D 2.1 uses ShapeVAE, SDF decoding, and surface extraction; its documentation includes sampling strategies for sharp features. This directs attention to sampling, compression, and decoding rather than assuming that any SDF-based output must be rounded.[3] FlexiCubes investigates optimizable isosurface-extraction parameters, illustrating that surface extraction itself can affect final mesh quality.[4]

TRELLIS.2’s O-Voxel combines flexible dual grids with sparse local attributes to address open surfaces and complex topology, organizing material information at geometric locations.[5] However, its paper’s appendix explicitly discusses sub-voxel structural aliasing: when closely spaced surfaces fall within the same local cell, geometry can still be compromised and materials averaged.[6] A different representation can therefore relieve particular bottlenecks, but cannot remove the limitations of input evidence and discretization.

Choose a representation using the same set of failure cases: are thin parts thickened, holes sealed, repeated ribs fused, and open boundaries preserved? An encode–decode–export round trip on known geometry is particularly useful. If the round trip already loses a type of detail, prioritize the representation and compression chain. If the round trip preserves it but conditional generation does not, investigate image conditioning, the training distribution, and the generation stage. This separates limitations in generation capability from limitations in representational capacity.

05Geometry repair: assign distinct roles to depth, normal, and sharp-edge constraints

Similar colors do not establish a correct surface. Wonder3D jointly generates multiview colors and normals, then constrains the surface through normal fusion; Unique3D further investigates high-resolution multiview generation and direct mesh optimization.[7][8] For engineering implementations, the transferable lesson is how to organize cross-view constraints. Model-predicted normals can also contradict one another, so not every pixel should be treated as an equally reliable measurement.

For a controllable reconstruction or refinement stage, one can formulate L = λs·Ls + λd·Ld + λn·Ln + λe·Le + λr·Lr. Here, Ls compares silhouettes, Ld compares depth within valid regions, Ln compares normal directions in a common coordinate system, Le constrains verified structural edges, and Lr suppresses unnecessary surface noise. This is an engineering formulation proposed in this report, not the original loss function from a cited paper. Terms must be normalized for scale, and weights established through comparative validation.

For example, a confidence-weighted directional error for normals can take the form Σ qᵢ·ρ(1 − nᵢ·n̂ᵢ) / Σ qᵢ, where nᵢ and n̂ᵢ are unit normals, qᵢ is a reliability weight, and ρ is a robust penalty. Visibility, occlusion, normal orientation, and camera-coordinate transformations must be handled before applying it; regions with Σ qᵢ = 0 are excluded from this term. Normals may legitimately be discontinuous across a sharp edge, so cross-edge consistency should not be imposed as a global smoothing objective.

Camera errors can masquerade as geometric errors. Near the optical axis, for small yaw or pitch errors, pixel displacement in the corresponding direction can be roughly estimated as δu ≈ f_px·δθ, with the angle in radians. At f_px = 1000, an orientation error of 0.2° corresponds to approximately 3.49 pixels, potentially wider than the groove being reconstructed. Increasing normal-supervision weight under these conditions may create surface ripples to accommodate incorrect projections. Calibrate the camera before optimizing geometry.

Sharp-edge treatment should also respect design intent. Split face normals and weighted vertex normals alter shading only; missing physical edges require mesh changes, while real bevels require a finite radius. Sharpening every edge distorts injection-molded parts intended to be rounded. For standard holes, planar interfaces, and repeated grilles, parametric reconstruction from known dimensions is usually easier to validate than repeated global sharpening of an uncertain surface.

06Material separation: why a sharp texture can still look like clay

Photographic pixels contain illumination. If highlights and shadows are projected directly into base color, the old shadow remains when the light changes, while a new highlight is added on top. Even correct geometry can then appear heavy, waxy, or dirty. IntrinsicAnything studies prior-based decomposition of materials and lighting; Material Anything targets consistent PBR material generation for 3D objects. These approaches infer appearance parameters; they do not replace the repair of incorrect shapes.[9][10]

Hunyuan3D-Paint 2.1 provides a documented example: its paper describes multiview generation of albedo, metallic, and roughness, cross-branch and cross-view alignment, and training choices intended to reduce lighting contamination. Geometry-rendered normals serve as conditioning information; this does not justify claiming that the model delivers an independent normal map.[3] When choosing a tool, verify each actual output and its meaning, rather than merely checking whether the interface says “PBR.”

Roughness controls the reflection distribution of microfacets; metallic corresponds to material properties in a metal–dielectric model. Neither is a general-purpose sharpness control.[11] Lowering roughness globally can make plastic, rubber, and metal all appear shinier, while exposing normal-field ripples that were previously inconspicuous. First establish interpretable regions based on components and materials, then inspect whether the same component responds plausibly to different lighting. A genuinely matte material should not be turned into a mirror to make it appear more premium.

For each critical component, this report recommends keeping three comparisons: the original generated material, a controlled reference material substituted without other changes, and the re-estimated material. With geometry fixed, if the reference material removes the clay-like appearance, prioritize material separation or import behavior. If the surface remains rounded, return to geometry. This experiment identifies the source of improvement more clearly than regenerating shape and texture simultaneously.

07Textures and export: a 4K label cannot replace local sampling and correct setup

Total texture resolution must be interpreted alongside UV area allocation. As an approximate budget, let T be the side length of a square texture, η its effective utilization, and A the covered surface area. Under uniform allocation with approximately negligible distortion, equivalent average linear density is approximately ρ = √(η·T²/A). With A = 300000 mm², T = 2048, and η = 0.6, density is about 2.90 texel/mm; a 0.2 mm line receives only about 0.58 texel. Doubling the texture side length raises this to only about 1.16 texel. This is a constructed estimate; actual local UV distortion and allocation can produce substantially greater variation.

Start with a checkerboard texture to inspect local stretching, seams, and texel density on the most important components. Then decide whether to unwrap again, reallocate UV space, or increase texture resolution. Bake after the target topology and normal conventions are stable. Check whether projection rays hit neighboring thin parts, whether UV islands have sufficient padding, and whether colors bleed at the target mip level. Fine scratches may be represented by normal or material maps; notches that change the outer silhouette or create real occlusion should remain geometric.

glTF 2.0 specifies that base-color RGB is interpreted as sRGB, while metallic-roughness and normal textures contain linear data. In metallicRoughnessTexture, the G channel stores roughness and the B channel stores metallic. Normal maps also depend on the correct tangent basis.[12] Incorrect color spaces, channels, or tangent conventions can give the same data different highlights and apparent relief. Validate import using explicit test spheres, planes, and known grooves, rather than repeatedly adjusting parameters against a single product screenshot.

The presence of an attribute and its correct display are separate questions. TRELLIS.2’s official export example notes that GLB uses OPAQUE mode by default; even when a texture retains alpha, transparent display still needs to be enabled correctly.[13] Alpha-based coverage is also not equivalent to full transmission, refraction, and volumetric absorption; transparent-part appearance depends on the target material model. Reimporting the final GLB and repeating the diagnostics is an essential part of delivery.

08Trace information loss by separating post-processing into comparable stages

Keep four clearly distinguished versions: the raw model or decoder output, the extracted mesh, the remeshed and decimated result, and the delivery file with UVs and baked materials. At each stage, save silhouettes, depth, and normal images under the same camera set. If the raw mesh has ventilation holes and the final file does not, locate the responsible post-processing step before attributing the result to an inadequate generation model.

Remeshing can improve triangle distribution, but may also merge neighboring surfaces or round sharp corners. Decimation must protect silhouettes, part boundaries, and material seams; a total polygon target alone is insufficient. Whether detail from a high-resolution model can be transferred to a low-polygon version through normal baking depends on whether that detail affects silhouette and occlusion. Features that cannot be transferred this way should retain geometry, or the intended use should be explicitly reduced.

Normalized silhouette error e_s = d_boundary / D_px can record local degradation, where d_boundary is silhouette boundary distance under an agreed statistic and D_px is the object’s image-space scale. Retain absolute pixel error and critical-region results as well, so normalization by a large object does not conceal loss of small parts. This metric is not a manufacturing tolerance and does not replace 3D measurement; its purpose is to locate the stage at which the same asset begins to deteriorate.

The same decomposition applies when training a proprietary model: first test data-conversion loss on reference assets, then encode–decode loss, and finally conditional generation. Audit cleaning and annotation for thin parts, openings, sharp edges, and adjacent materials separately. If preprocessing has already closed every opening, simply increasing generator scale cannot reliably relearn structures systematically removed from the data.

09Constructed example: diagnosing an industrial control box with a grille

The following example illustrates a decision process and contains no real customer data or model-test results. Assume a control box has a bounding extent of 240 mm, side-grille gaps of 1.2 mm, and a designed housing bevel radius of 0.6 mm. It consists of a coated metal housing, plastic knobs, and rubber feet. The target is close-up web viewing and design review. Openings, major proportions, and material differentiation must hold, without claiming delivery of manufacturing-grade CAD.

Suppose the 240 mm physical width of the inspected face spans 80% of the width of a 512-pixel input image, with that face approximately front-facing. A 1.2 mm gap then spans only about 512 × 0.8 × 1.2 / 240 = 2.05 pixels. At an output axial resolution of 512, Δ is approximately 0.469 mm: the gap spans about 2.56 cells, and the 0.6 mm bevel radius spans approximately 1.28 cells. These are details requiring close inspection; the numbers alone do not establish a pass or fail.

First compare the source image with the extracted foreground to confirm that resizing or segmentation has not removed the grille gaps; then capture additional views of that region or introduce design dimensions. After geometry generation, inspect the side silhouette, oblique-view occlusion, and whether the openings actually pass through. If the gaps are already sealed, increasing texture resolution is irrelevant. Consider a more suitable representation or refinement method; a regular grille can also be rebuilt locally from confirmed dimensions.

Next, fix the mesh and substitute controlled reference materials for the three material regions, checking for residual highlights and shadows in the original color. If broad, bulging highlights persist on the housing, compare the actual geometry with the vertex normals. Finally, decimate, bake, and export in stages, retesting in the target web viewer. Repair is complete only when the delivered file preserves the grille, bevels, and material responses.

10Define a validation protocol: do not let average similarity hide critical failures

Organize the test set into hard-surface objects, thin parts and openings, small repetitive structures, mixed materials, and highly reflective objects, with a separate group lacking clear rear-view observations. Mark a small number of use-critical regions in each sample. CLIP or other semantic similarity measures can help verify object identity, but cannot prove that a particular hole passes through. Overall image similarity may also be dominated by large areas of correct background or flat surfaces, concealing local failures.

Where reference geometry exists, record units, registration methods, and scale handling before combining symmetric surface distance, normal-angle error, and threshold-dependent F-score. Where physical dimensions matter, freely scaling during registration must not silently remove dimensional error. Check survival of thin parts, connectivity, and expected boundaries separately. Whether an open surface may contain non-manifold structure depends on the intended use; watertightness should not be imposed on every visual asset.

Without reference geometry, validation can establish multiview consistency, agreement with known dimensions, editability, and usability in the target environment. It cannot establish recovery of the true rear surface or justify a geometric-accuracy claim. Material evaluation should retain camera or lighting conditions not used during fitting, with exposure and color management fixed; otherwise, an improvement from changing environment lighting may be misrecorded as a model improvement.

At a minimum, separate geometry changes, material changes, and import corrections in comparative experiments. Keep model versions, input preprocessing, and random seeds traceable. Compare generation success rates over a fixed set of seeds rather than a single best result, and include failed samples. For each approach, report the number of critical regions passing validation, manual repair time, inference and processing cost, file size, and behavior on the target device. Agree on defects that block delivery before comparing average quality.

A workable minimum validation package includes input evidence and scale references, model and parameter versions, assets from each processing stage, critical-region screenshots, fixed test scenes, and failure descriptions. Set thresholds according to intended use, viewing distance, and the cost of error. Marketing previews, design reviews, interactive applications, and manufacturing preparation should not share an unsupported universal “passing score.”

• Geometry: check silhouette, holes, thin parts, sharp edges, and dimensions separately.
• Materials: hold geometry fixed and compare unlit color with responses under different lighting.
• Delivery: inspect the final file in the target application and on the target device.
• Cost: include retries, manual repair, and failed samples in the cost per usable asset.

11Implementation priorities: repair the earliest information loss before considering retraining

When an existing model produces reasonable geometry, first investigate material separation, UVs, normals, and engine import. These issues generally do not require retraining the shape model. If critical structures are visible in the input but repeatedly disappear from raw generation results, use the same failure set to compare better-suited representations, local reconstruction, or geometric constraints. Consider investment in domain-specific training only after confirming that the data and representation can preserve the target detail.

Training data should preserve relationships among geometry, materials, and capture conditions. Avoid treating illuminated RGB images uniformly as albedo ground truth. Split datasets by object and near-duplicate asset to prevent different views of the same object from entering the test set. Evaluation sets can deliberately include more thin parts, sharp edges, and objects with openings, but training sampling should not indiscriminately amplify all high-frequency signals, or scan noise may also be learned as detail.

When repairing a class of structures within the automated pipeline consistently costs more than rebuilding it locally, retain a hybrid workflow: the model supplies overall form and candidate appearance, rule-based modeling supplies known dimensions and repeated structures, and material processing supplies relightable surfaces. Evaluate the total delivery cost of a validated asset, rather than single-run generation speed or the percentage of automation.

IDENIFE’s image-to-3D capability page frames the objective as making generated assets usable in subsequent design work. Applied to the clay-like look, this means bringing input evidence, geometric checks, material tests, and final-file validation into one delivery process. For a particular product, first establish practical boundaries with representative samples, then discuss integration scope. This report provides no measured evidence sufficient to promise a specific model accuracy or manufacturing tolerance.