BlendGen

BlendGen is a synthetic data generation tool built on Blender, developed during my CIFRE PhD at SEGULA Technologies and the CNRS Hubert Curien Laboratory. It produces controlled RGB-D datasets with automatically generated annotations.

It is released as a Blender add-on through the Hubert Curien Laboratory GitLab, and described in a journal paper published in 2023.

Supervised computer vision needs annotated data, and depth-annotated data is particularly expensive to collect. In industrial settings the constraint is sharper still: the objects and scenes of interest may be rare, may not yet exist physically, or may be impractical to photograph exhaustively.

Manual annotation of depth is not merely tedious; for many scenes it is not accurate enough to be useful as ground truth.

• RGB-D ground truth is difficult and costly to obtain physically
• Rare configurations are underrepresented in collected datasets
• Annotation quality limits the ceiling of supervised performance
• Experiments are hard to reproduce when the dataset is fixed and opaque

• Generate RGB-D image pairs with exact, automatically derived ground truth
• Make scene composition parameterisable so datasets are configuration, not manual work
• Support domain randomisation over lighting, materials, pose and background
• Guarantee reproducibility: the same configuration must produce the same dataset
• Produce annotations in formats consumable directly by training pipelines

• Rendering time bounds how large a dataset can practically be
• Synthetic-to-real domain gap must be actively managed, not ignored
• Scene configuration must stay usable by researchers who are not Blender specialists
• Output must integrate with existing training code without conversion scripts

1. 01

   Scene Configuration

   Declarative description of objects, materials, ranges

2. 02

   Randomisation Engine

   Samples pose, lighting, materials and background

3. 03

   Blender Rendering

   Automated headless rendering

4. 04

   RGB + Depth + Annotation Export

   Paired outputs with exact ground truth

5. 05

   Dataset Assembly

   Splits, manifests and reproducibility metadata

• Configuration declares the object set, the sampling ranges and the output specification
• A seeded random generator samples a concrete scene from those ranges
• Blender renders the scene headlessly, producing colour and depth passes
• Annotations are derived from the scene graph rather than inferred from the image
• Dataset manifests record the seed and configuration used, so any sample can be regenerated

• Headless batch rendering to remove interactive overhead
• Reuse of loaded assets across samples to avoid repeated scene setup
• Render settings tuned to the quality actually needed for training rather than for visual fidelity
• Parallel rendering across processes

• Distributed as a Blender add-on, driven by configuration files
• Used to generate training data for the model-optimization work carried out during the PhD
• Datasets produced offline and consumed by standard training pipelines

• Seeded generation, so a dataset is reproducible from its configuration
• Configuration validation before rendering begins, rather than failing mid-batch
• Per-sample failures isolated so one bad scene does not invalidate a run
• Manifest files recording exactly what was generated and how

The relevant question for a synthetic data tool is not rendering speed but whether models trained on its output transfer to real data. Evaluation therefore compares models trained on synthetic data, real data, and mixtures, on a held-out real evaluation set.

Quantitative results from this work are tied to the associated doctoral research and industrial context. No figures are published here.

Benchmarking in progress. Results will be published with the hardware, model, runtime and measurement conditions that produced them.

• Reproducibility must be built in from the start; a dataset you cannot regenerate is a dataset you cannot debug.
• Domain randomisation is more effective than pursuing photorealism when render budget is the binding constraint.
• Deriving annotations from the scene graph rather than from the rendered image removes an entire class of label noise.
• The tool became genuinely useful only once generating a dataset stopped requiring Blender knowledge.

• Broader sensor models beyond the RGB-D configuration used
• Systematic study of which randomisation axes matter most per task
• Tighter coupling between generation configuration and training experiment tracking