How to perform a 3D Segmentation in Blender 2.82

The availability of 3D shape data significantly increased over the past years due to advances in 3D capturing sensors and 3D modeling tools such as Blender. This naturally leads to a growing interest in the high-level understanding of 3D shapes. Specifically the part segmentation of 3D shapes gives valuable insights into the characteristics of an object by dividing it into its meaningful parts.

Several deep learning frameworks exist to perform a part segmentation of 3D objects. One popular example is MeshCNN [1], which works directly on the edges of 3D mesh representations. To train MeshCNN it is necessary to provide a manual ground truth segmentation that contains all edge labels ordered by their edge ID.

Within this article, I will show how I performed a manual part segmentation within Blender 2.82 and used it as a ground truth for training MeshCNN. Before diving into the step-by-step description, let’s take a quick look at the input that MeshCNN requires.

Naturally, it is necessary to provide all input objects as .obj files for training and testing the model. For each object, there should be a corresponding .eseg file, which collects the class labels per edge ordered after their edge IDs. Each row thereby provides the class of the edge with an edge ID equal to the row number. The .eseg file is used to train the network and is extracted from Blender after performing the part segmentation.

Furthermore, MeshCNN requires one .seseg file per object to compute the test accuracy. It allows that each edge may have more than one correct segmentation class. This is allows smooth transitions between the segmentation classes. Each .seseg file provides a value for all segmentation classes for each edge. Each column with a value greater than zero indicates a correct segmentation class. The values derive from the segmentation labels of the neighbors of each edge. The .seseg files are created after manual part segmentation within Blender.

In order to perform a manual part segmentation of an object in Blender, it is necessary to import the objects into Blender. Thereby I would suggest to keep the Vertex Order of your Object by highlighting “Keep Vert Order” under the Geometry section, when importing the object file.

Now we are ready to perform the actual part segmentation. For this we will utilize Blender vertex groups. Each vertex group corresponds to one segmentation labels. To assign vertices to a vertex group select a select tool and highlight all vertices that belong to one class. I personally prefer to use the Circle Select tool with the shortcut “c”. Once all vertices of one label are collected, assign them to their dedicated vertex group. This is done by selecting the correct group and press Assign.

There are multiple hacks to accelerate the manual segmentation. First of all it is possible to select an X-ray view of the object. This view lets you select vertices through the whole object and not only the front side. c

Furthermore, it is possible to remove groups from other groups. For the chore segmentation class for example, I assigned all vertices of the object to vertex group “chore”, afterwards I deselected this group and selected the other groups Legs, Feet, Arms, Head. Finally I removed all groups from the chore group to end up with only the chore vertices.

It is worth noting, that the part segmentation remains when you are upsampling or downsampling the mesh within Blender. Any alteration is directly transferred to the vertex groups and can be directly extracted afterwards.

I hope this description helps you to create a manual part segmentation for MeshCNN or other part segmentation frameworks.

[1] Hanocka, R., Hertz, A., Fish, N., Giryes, R., Fleishman, S., Cohen-Or, D., 2019. Meshcnn: a network with an edge. ACM Transactionson Graphics (TOG) 38, 1–12

Related posts:

Making a decision on personal growth and development is certainly a big step. Based on my experience both as a client and a life coach, I have identified 3 traps that can hinder self-development. I…

We live in a world of buzzwords and specialized nomenclature. Sometimes we may have heard a word or concept flung around several times before realizing we’re not really sure what it means. Instead…

If you think you might be tomorrow’s Zuckerberg or Bill Gates, when the deeds of people like Steve Jobs fit your dreams, or the technology lights up the fire in your stomach, there are few things in…