»Ê¼Ò»ªÈË

ARC's Research Software Engineers worked with WEISS to develop a Blender add-on to randomise relevant parameters for the generation of datasets for polyp detection within the colon during surgery.

Background

Blender isÌýfree and open-source softwareÌýfor 3D geometry rendering.ÌýUses include modelling, simulation, animation, virtual reality applications, and more recently synthetic datasets generation.ÌýThis last application is of particular interest in the field of medical imaging, where often there is limited real data that can be used to train machine learning models.ÌýBy creatingÌýlarge amountsÌýof synthetic but realistic data, we can improve the performance of models in tasks such asÌý.ÌýSynthetic data generation has other advantages since using tools like Blender gives us more control andÌýwe can generate a variety of ground truth data from segmentation masks to optic flowÌýfields,Ìýwhich in real data would beÌývery challengingÌýto generate or would involve extensiveÌýtime consumingÌýmanual labelling.ÌýAnother advantage ofÌýthis approach is that often we can easily scale up our synthetic datasets by randomising parameters of the modelled 3D geometry. There can be challenges to make the data realistic andÌýrepresentative of theÌýreal data.

The ProblemÌý

The aim was to develop an add-on that would help researchers and medical imaging experts determine which range of parameter values make realistic synthetic images.ÌýPrior to the project, the dataset generation involved a more laborious process of manually creating scenes in Blender with parameters changed manually for introducing variation in the datasets. A more efficient process was neededÌýduring the prototyping of synthetic dataset generationÌýtoÌýdecide what range of parameters make sense visually,ÌýandÌýtherefore in the future, toÌýmore easilyÌýextend to other use cases.Ìý

What we did

In collaboration with the UCLÌýWellcomeÌý/ EPSRC Centre for Interventional and Surgical Sciences (WEISS), and , who are research software engineers from ARC, developed a Blender add-on to randomise relevant parameters for the generation of datasets for polyp detection within the colon.ÌýThe add-on was originally developed to render a highly diverse and (near) photo-realistic synthetic dataset of laparoscopic surgery camera views. To replicate the different camera positions used in surgery as well as the shape and appearance of the tissues, we focused on randomising three main components of the scene: camera transforms (camera orientation and location), geometry and materials. However, we allowed for more flexibility beyond these 3 main groups of parameters, implementing utilities to randomise other user-defined properties. The software also allows the following features: 1) setting the minimum and maximum bounds through an input file, 2) setting a randomisation seed for reproducibility, 3) exporting output parameters for a chosen number of frames to an output file. The add-on includes testing through Pytest, documentationÌýfor users and developers,Ìýexample input and output filesÌýandÌýa sample Blender scene.

The OutcomesÌý

Version 1.0.0 of the Blender Randomiser is available under a BSD 3-Clause License. The GitHub repo isÌýÌýwhere the software can be downloaded and installed with instructions provided on how to use the add-on. Examples of what can be produced in Blender can be found at theÌýÌý(N.B. these examples were produced manually prior to completion of this project).Ìý

Developer notes are also available to allow contributions.Ìý

Links

• UCL WellcomeÌý/ EPSRC Centre for Interventional and Surgical Sciences (WEISS)

---