A D VANCED CLUSTERING OF ARCHITECTURAL GEOMETRIC ORNAMENTS USING SMALL SCALE MACHINE LEARNING, CASE STUDY OF ILKHANID GEOMETRIC PATTERNS *

: Classification is an essential step for architectural historians for better understanding and typology of cultural heritage. This research aims to automatically cluster geometric ornaments of the Ilkhanid period in Iran through using machine learning It examines the application of advanced computer science tools and methods in analysis of architectural heritage by searching the possibility of clustering ornament images with machine learning into different clusters. After examining case studies of mosques, tombs or other buildings in Iran from the Ilkhanid period (1256-1335 CE), 231 images from 36 existing buildings were chosen, and after editing images, an inventory was created based on characterization of each ornament (image) containing: Buildings’ Name, Region, Construction Date, Functionality, Repetition of Motifs, Material, Ornamental Types, Design Complexity, Dominant Colour, Star-Number, Geometric Shape, Geometric Lines, and Geometric Pieces. Next, these images were analysed with small-scale machine learning with the help of the visual programming toolbox Orange (http://orange.biolab.si). The results containing image groups (machine clustered images) were tested with the characterization table of ornaments, and groups of ornaments that represents a style is introduced.


INTRODUCTION
In recent years, advanced methods in computer science have been used to discover knowledge of architectural heritage.One of the analytic methods is machine learning in computer vision, which provides advanced high-speed analysis with repeatable selections and improves the results of clustering multiple data in a short period of time.One important research in the field of architectural heritage is to understand categories of styles of buildings based on formal features such as decoration, spatial organization, shape of composition of materials, etc. Machine learning can help to find detailed categories of cultural heritage based on common features and improve the knowledge discovery of architectural heritage, which may lead to new discovery of less known styles.Decorations are distinctive features of architectural heritage style, especially in Islamic culture.Knowing the rich styles of Iranian Islamic architectural decorations requires categorizing different types based on various characteristics such as materials, motifs, shapes, etc. by using advanced methods.Ilkhanid period (1335-1256 A.D.) can be considered the most productive period of Iranian architecture with sophisticated ornaments and rich decorations (Carboni, Masuya, 1993, Bloom, 1997).The aim of this research is the advanced clustering of Ilkhanid decorations with a focus on geometric patterns using a small-scale machine learning method based on images of decorations.First, samples of geometric decorations were collected based on their images.When the database is ready by filed survey and available resources, the analysis is started.An unsupervised machine learning process for image clustering is used.The results were analysed in different steps, including first, comparing the clustering results (automatic) with the characteristics table (manual).Second, automatic clustering groups analysis with feature table.Finally, the automatically clustered image groups were evaluated in subsequent layers.

CLASSIFICATION OF ARCHITECTURAL DECORATION
There are many features that help scientists classify architectural decorations into different groups.Most historians of art and architecture have presented categories based on the historical period, which are introduced with the names of rulers or other important builders.(Pope, 1939;Zomorshidi, 1986;El-Said, Parman, 1976;Godard, 1965, etc.).Types of patterns are important features of classification as well.The most common ones are plant motifs (Kakhaki, Taghavinejad, 2016), inscriptions, and geometric motifs.Geometric decorations are also examined based on different motifs.For example, the number of sides of the star (central polygon) for motifs with a star pattern inside it is investigated as the evolution of geometric motifs in Islamic art and architecture (El-Said, Parman, 1976;Abdullahi, 2013, Taghavinejad, 2017).2017).The classification features are general and sub features and more detailed clusters that can contribute to recognizing new styles are not introduced, a task that this research is trying to fulfil.Among different categories of decorations, Islamic geometric decorations have varied patterns that can be recognized by advanced computer-based analysis and clustering.They are based on composition of intersecting lines which forms geometric patterns in repetitive modules called "Gereh" or "knot".The term "Gereh-sāzī" or "making knot" refers to a form of geometric interlaced strapwork ornament or interlocking star-polygon pattern that is commonly found in architecture and the minor arts throughout the Islamic world."Gereh" in Persian means "knot" that may refer to knots of lines (Milwright, 2001) (Sarhangi, 2012) (figure 2).One very important motif in geometric decorations which is always at the centre of a repetitive pattern of intersecting lines is "Shamsa".The word, literally means "little sun" is considered as the sun at the center of weaving lines of geometric patterns and shows the importance of light and sun in decorations.(Lee, 1987;Raeeszadeh and Mofid, 2011).
Here the question is how these distinctive features of Islamic geometric decorations such as Gereh and Shamsa can contribute to clustering of highly varied ornaments and can help to better recognize new styles of Ilkhanid architecture.

CLASSIFICATION OF GEOMETRIC PATTERNS AND IMAGE COLLECTION
The research has implemented the image of an ornament as basic data for classification.For each image different types of attributes are collected as internal or external features.building's name, date of construction, building's location and building functionality are external associated attributes.Color that represents material, and type of ornament that includes geometric or not geometric and the semantics of the geometry such as type, line, and repetitive pattern are the internal sttributes, Figure 3 shows Star-Number of Geometric patterns (star points or polygon side number).The image data-set of each building (selected as figure 1) was collected both after library study and field survey and photography of monuments.From each ornament, the repetitive star-shaped unit was separated in an square images with similar image sizes.The total of 230 images and sample of ornaments were selected for analysis based on the defined features as shown in table 1.

HIERARCHICAL CLUSTERING
In data mining and statistics, hierarchical clustering (also called hierarchical cluster analysis or HCA) is a method of cluster analysis which try to build a hierarchy of clusters based on common features.(Rokach, Maimon, 2006).This research has chosen the Cosine similarity as a measure of similarity between any two elements of X, and the ward distance as the linkage distance for defining the sub-set distance.Here, an unsupervised machine learning algorithm was used for clustering images with the help of the visual programming toolbox Orange (http://orange.biolab.si).First all images were imported and the images pixels were used for automatic clustering (1).Later, all the images were embedded and transferred to a learning algorithm named Google's inception v3 and a vector of numbers was obtained for each image (2).InceptionV3 is Google's deep neural network for image recognition which is trained on the ImageNet data set (Szegedy et al., 2016).The next step was to measure distances between images' vector data

Semantic of Clusters
In this phase, each of the 21 groups was analyzed separately with their features in the characterization table to find the similarity percentage.Therefore, in every column of features, the most repetitive item was chosen, and its repetition percentage was calculated.Then an average of all column's percentage is determined as the similarity percentage of the groups.
The similarity percentage of groups is between 53 and 78, mostly between 60 and 70.Some of these groups might be able to divide into smaller groups.Studying clustered groups shows that some of these groups have the ability to become a particular style, even though they are not comprehensive and some of the images that can be attributed to them are placed in other groups.
Here 2 importnt groups are introduced as follows.
Table 1: classification features of Geometric decorations in this research

Introducing cluster C02
This group contains 5 images with a similarity percentage of 78.
The effective features that their similarity is above average in the group are Rounded date of construction (Second), Ornamental Types (Geometric Only), Design Complexity (Complicated), Repetition of Motifs (Motif-Set Repetition), Geometric Shape (Gereh), and Geometric Pieces (No-pieces) as shown in figure 5.This group contains images with a complex design, which is created from two series of geometric lines that generate one star-shaped motif named "Gereh" in Islamic art.The main feature of this style is its great complexity that is perceptible in both the shape of motif and star-number.The star-numbers are mostly 8 and 6 which are more complex than 4 or 3 but there is a 7 star which is so rare and hard to create geometrically at the time of construction.The combination of three colors of cyan, cobalt-blue, and white in the intricated embossed lines has made this style attractive.The lines in this group are not completely sharp-cornered and there are little arches in the joints that make it similar to Eslimi shapes.According to the above description, this group has features to be considered as a unique style which can be named as "Intricated Geometric lines".

Introducing cluster C18
This group contains 13 images with a similarity percentage of 72.The effective features that make their similarity above average in the group are Rounded Date of Construction (Second), Design Complexity (Complicated), Repetition of Motifs (Motif-Set Repetition), Geometric Shape (Gereh), Geometric Lines (Embossed lines), and Geometric Pieces (Nopieces) (figure 5).The ornament's shapes in this group are all created with embossed lines and there are almost no pieces in them, one of them contains colored lines which is embossed too.They are all complicated ornaments with the Gereh shape except the one that is a 4-sided shape combined with the Eslimi (a set of herbal / vegetation shapes and designs such as curved stems, buds, flowers that start with a base and grow around that base (Esfahani, 2014)) (table 2).The star-number of 8 is more common and the plaster is used more as material.The location of ornaments in the building which was mostly not noticeable for the altar, is more common for these ornaments, that might be because of its sacred belief.These ornaments are from the first and second periods of rounded dating, the only one from the third period is most probably from an older time of the building construction.This explains that this style is older and effects on other ornamental styles of this period (figure 6).

ASSESSMENT
The proposed approach to image mining uses visual analytics (24 in orange) that combines interactive visualizations and automated data analysis, including machine learning.For the assessment of results in this research rerunning the program by changed names was used.All image names were changed and the characterization table was deleted from the program to prevent any data similarity and then the program was re runed.Images were loaded, embedded, and their distance was shown on a hierarchical dendrogram; the results were exactly the same.Below the programming process, hierarchical dendrogram, and group C03 were shown as an instance to show that the images are exactly the same even with different names or image codes (figure 7).

CONCLUSION
This research has used advanced cultering by machine learning tools to recognize the styles of architectural decorations of the Islamic period from the images.The groups of hierarchical clustering were analyzed and described by their characterization table and located on the map.A similarity percentage table with their above-average features was produced to analyze the hierarchical image clustering.There are also other images that do not place in a group or subgroup as individual styles but they can be considered as a particular style too, However, these clustered groups of ornaments can be used to recognize the styles and help to further analyze the ornament's characterization and comprehensivity.

Figure 1 .
Figure 1.Map of Selected monuments of the Ilkhanid era that include star-shape geometric decorations (based on the list of Iran's National Heritage List) One of the good examples of classification based on different features was done by Taghavinejad, who used other classification methods such as the complexity of motifs, linestypes, combining motifs with other patterns, etc. (Taghavinejad,2017).The classification features are general and sub features and more detailed clusters that can contribute to recognizing new styles are not introduced, a task that this research is trying to fulfil.Among different categories of decorations, Islamic geometric decorations have varied patterns that can be recognized by advanced computer-based analysis and clustering.They are based on composition of intersecting lines which forms geometric patterns in repetitive modules called "Gereh" or "knot".The term "Gereh-sāzī" or "making knot" refers to a form of geometric interlaced strapwork ornament or interlocking star-polygon pattern that is commonly found in architecture and the minor arts throughout the Islamic world."Gereh" in Persian means "knot" that may refer to knots of lines(Milwright, 2001)   (Sarhangi, 2012) (figure2).

Figure 2 :
Figure 2: One kind of Traditional Gereh and its geometric units (Alat).

Figure 3 .
Figure 3. Star-Number of Geometric patterns (star points or polygon side number).
(3).The distance metric was the cosine (a trigonometric function) of the angle between two vectors of images.The resulting distance matrix was fed further to Hierarchical Clustering for uncovering groups in the data, (4) which computes hierarchical clustering types of objects from the matrix of distances and shows a corresponding dendrogram.Which represents the nested grouping of images and similarity levels at which groups change.A clustering of the data is obtained by cutting the dendrogram at the desired similarity level(Rokach &, Maimon 2005).Based on testing different similarity levels and grouping on the dendrogram, 21 clusters were obtained.The selected clusters on dendrogram can be seen on image viewer widget (5) (Figure4).

Figure 4 :
Figure 4: Process of clustering images in Orange-software

Figure 6 :
Figure 6: Group C18 images The main groups can represent a style of architectural decoration as follows: 1. Cut tile: This style is made of colored cut tiles that have geometrically sharper stars.2. Square and rational lines: It consists of square lines (pixels) and construction line.Its star is drawn based on the number 4 and 8 and has simple designs.They are used for writings or calligraphy.3. Nested geometry: This design consists of two series of geometric lines in a field that are combined together.The main feature of this style is its high complexity, because this complexity can be seen both in the role of this pattern and in the numbers of Shamseh.4. Large one-piece and seven-color tile: This design is made up of larger one-piece tiles and pieces, which also use slime and animal designs.Most of them are simple and have a combination of 4 and 8 that are put together and this work has a high execution speed.There are also some subgroups in them that can be considered as particular styles or related to other styles.Such as follows:• C18: Intricated Plasterwork -Monocolor Gereh • C06: Sharp Musaic Tiles with Star-Number of 6 (C05) • C12: Intricated Geometric Lines with Thick lines (C05) • C21: Royal or Main (C15) • C14: Royal or Main used at Ceilings (C15) • C04: Sahrp Musaic Tiles (C05) • C07: Sharp Musaic Tiles with simple design and small size (C05) • C20: Royal or Main (C15) -Colored Embossed Lines), C13 (Diamond Brickwork (C09) • C19: Geometric-Eslimi Gereh -Star in a Circle • C01: Geometric-Eslimi Tiles), C10 (Unknown Gereh -Monocolor Polygon • C11: Monocolor Polygon (C10) -Intricated Plasterwork (C18) -Geometric Calligraphy), and C08 (Intricated Geo.Lines (C02) -Royal or Main (C15) -Square Shape Calligraphy.

Figure 7 .
Figure 7. Process of re running the model with changed names in orange

Table 2 :
Ornamental features of group C18 based on the characteristic ornamental table