TIAN Huaiwen and QIN Shengfeng

?

Exploring Local Regularities for 3D Object Recognition

TIAN Huaiwen1and QIN Shengfeng2,*

1 College of Mechanical Engineering, Southwest Jiaotong University, Chengdu 610031, China;2 Department of Design, Northumbria University, Newcastle upon Type, NE1 8ST, UK

In order to find better simplicity measurements for 3D object recognition,a new set of local regularities is developed and tested in a stepwise 3D reconstruction method, including localized minimizing standard deviation of angles(L-MSDA), localized minimizing standard deviation of segment magnitudes(L-MSDSM), localized minimum standard deviation of areas of child faces (L-MSDAF), localized minimum sum of segment magnitudes of common edges (L-MSSM), and localized minimum sum of areas of child face (L-MSAF). Based on their effectiveness measurements in terms of form and size distortions, it is found that when two local regularities: L-MSDA and L-MSDSM are combined together, they can produce better performance. In addition, the best weightings for them to work together are identified as 10% for L-MSDSM and 90% for L-MSDA. The test results show that the combined usage of L-MSDA and L-MSDSM with identified weightings has a potential to be applied in other optimization based 3D recognition methods to improve their efficacy and robustness.

stepwise 3D reconstruction, localized regularities, 3D object recognition, polyhedral objects, line drawing

1 Introduction?

Toward a future manufacturing environment underpinnedby Industry 4.0 technologies, object recognition is aprecursor to many other important robotic tasks, includinggrasping, manipulation, inspection and assembly.

3D object recognition from a 2D line drawing is aresearch topic related to machine intelligence and computer vision[1–2] and sketch-based interface and modeling[3].

A line drawing can be converted from freehand sketches involving sketch tidy-up processing[4–5]. It can also be derived from image segmentation or edge detection[6]. Another source for line drawing is from CAD packages such as AutoCAD?.

Popular methods of 3D object recognition from 2D line drawing are optimization based inflation methods. Most of them assume that the face topology of a line drawing is known in advance.

In general, optimization-based inflation methods iteratively change depths to vertices and test interim results with a set of compliance functions involving various regularities. These methods treat the 3D object recognition from a 2D line drawing as a global optimization problem involving a large number of variables and a good number of image regularities with different weightings. Thus, the main problem with these methods is that they are computationally heavy and with no guarantee for a good result when dealing with a complex drawing. In addition, various regularities are applied globally and guides for setting up weightings for regularity are unclear due to the lack of knowledge about their relative importance.

The strategic solution to the problem has twofold. First, adopt a divide-and-conquer strategy to divide a whole task into multiple smaller and easy-to-solve ones at local level. Second, use most influential local regularities with effective weightings to reduce the problem complexity in optimization.

In Ref. [7], a divide-and-conquer strategy is applied to decompose a complex line drawing into component drawings or simpler ones and each simpler drawing can be recognized easily and locally. In Ref. [8], a level-by-level 3D reconstruction method is explored to divide a global optimization-based 3D reconstruction from a complex drawing into multiple local reconstructions from a subset of the drawing, based on its face/surface connection graph.

However, there is very little work on the development of local regularities and identification of their usages in terms of what regularities should be used together and how to set up their weightings.

In this paper, a stepwise 3D reconstruction method is used for exploring local regularities for recognizing a 3D polyhedral object from its 2D axonometric drawing. As a result, it is found that two local regularities: L-MSDA and L-MSDSM need to be used together as a combined regularity and their weightings are 10% for L-MSDSM and 90% for L-MSDA respectively. This combined local regularity has a potential to be applied in various optimization-based inflation methods in order to have better efficacy and robustness. A prototype platform on Matlab?has been developed for case studies and evaluation. The evaluation results demonstrate that the proposed method is a good way to explore various regularities for objects with resolvable representations[9]. The contributions of this paper include:

(1) Exploration and development of the localized regularities: L-MSDA, L-MSDSM, L-MSDAF, L-MSSM and L-MSAF and the development of a method to study their relative importance;

(2) Identification of the usage for combining L-MSDA and L-MSDSM together;

(3) Test cases from AutoCAD drawings.

The paper structure is as the following. The related work and the utilized stepwise reconstruction method are presented in sections 2 and 3 respectively. In section 4, the exploration and identification of key local regularities and their usages are followed up before case evaluations in section 5. Finally, conclusions are drawn in section 6.

2 Related Work

Early work on 3D recognition of an object from a line drawing is focused on line labelling[10–11], attempting to evaluate whether the drawing represents a valid solid object. An optimization-based inflation process has been then developed to reconstruct 3D objects.

MARILL[12]observed that human minds prefer a simple interpretation over a complex one. But, how to measure the simplicity of a recognized object is not clearly and well defined. In Ref. [12], the simplicity is measured by minimizing the differences among angles created between lines at junctions across the reconstructed object. It is termed as the minimizing the standard deviation of angles (MSDA). LECLERE and FISCHLER[13]added face planarity, and LIPSON and SHPITALNI[14]added more image regularities into an optimization process, and demonstrated the ability to recognize a wide range of objects. ERIC and BROWN[15]replaced the MSDA with minimizing the standard deviation of segment magnitudes (MSDSM) and produced better results for some cases compared with MSDA. This research suggested searching for alternative regularities to reflect the simplicity measurement, which inspired our work here.

VARLEY and MARTIN[16]found that the cubic corner property was particularly useful. A cubic corner is a junction where three mutually orthogonal planes meet, which can is derived and understood from MSDA. The relationship between a cubic corner in 3D and its 2D projection was established by PERKINS[17].

YONG and FANG[18]studied a direct 3D recognition method from a 2D drawing, requiring two sets of information from the drawing: a list of faces and a cubic corner. This research demonstrated that the use of face connections to trace down a reconstruction process could be an effective and accurate way to solve a 3D reconstruction problem.

For reducing the complexity of a 3D reconstruction, LIU, et al[19], considered MSDA and face planarity as two most important constraints. SUN, et al[20], reviewed the usages of various regularities in literature and recommended a set of key regularities for general 3D reconstruction based on the automatic relevance determination method. It is believed that the regularities are not equally informative in the simplicity measuring and some may deal with mainly to noise.

It is clear that many regularities have been used in previous research but their relative relationships are not known. This study demonstrates a way of studying their relationships or relative importance, exploring possible regularities further and identifying their usages.

3 Stepwise 3D Reconstruction Method

3.1 Principles of the stepwise 3D reconstruction method

An engineering object can be represented by an isometric drawing or oblique drawing with or without hidden lines (Fig. 1). The understanding of a 2D drawing into a 3D object can be a component-based method grounded on a theory of human image understanding[21]or a global optimization based method[13]. Interpreting a 3D object from a oblique drawing can be found in Refs. [22–24].

Fig.1. Axonometric projection

A component based method solves a problem in a stepwise fashion and might be quick and reliable because it finds solutions step-by-step and at each step, the problem is less complicated for solving. Thus, in this study we utilized a stepwise 3D reconstruction method.

This stepwise 3D reconstruction method is based on the theory of axonometric projection. Under an axonometric projection, a 3D point (,,) will produce a 2D projection point (,). Their relation is

where

1and2represent two rotation transformation matrices while3describes an orthographic projection. Under this transformation in Eq. (1), every 3D vertex(,,, 1) in the 3D object has a corresponding 2D point in the drawing represented as a vertex(,, 0,1) from=.

is not reversible. But when a constraint is introduced in each step, in a stepwise reconstruction process, saying that a point to be reconstructed is on a planar face (face regularity is applied explicitly here),is then converted into a reversible matrix and it can be used to support 3D reconstruction by a reverse transformation.

In generality, a planar face in 3D can be represented by Eq. (3), where () is the face normal andis the distance to the origin:

Now, if this face is used as the constraint,matrix can be updated as a reversible matrix: