INTRODUCTION

Sex determination of human bones is one of the important tasks that forensic anthropologists must perform for law enforcement agencies and coroners in the identification process. The sex assessment is crucial as age, stature, and ancestry are all dependent on sex (^{Kilmer & Garvin, 2020}). In previous studies, the pelvis and skull were the most reliable bones for sex determination (^{Klales et al., 2012}). According to the finding of those studies, the pelvis has the best performance for classifying sexes and is the least damaged bone (^{Mahadevappa & Shivalingaiah, 2017}). The highly sexually dimorphic features of the pelvis include subpubic concavity, pubic bone shape, greater sciatic notch, ventral arc, and posterior ilium. The accuracy ranges from 77 to 95.5 percent (^{Novak et al., 2012}; ^{Blake & Hartnett- McCann, 2018}), in which posterior ilium is the most sexually dimorphic. Therefore, the posterior portion of the ilium will be the subject for testing sex determination in the Thai population.

The posterior ilial portion was previously studied to assess its sex discriminant ability, which showed moderate to high accuracy for classifying sexes (Novak et al.; ^{Wescott, 2015}; ^{Karsten, 2018}). Karsten observed the preauricular sulcus for sex determination in American blacks and whites, and the accuracy was 75.8 percent. Males showed a short and narrow sulcus whereas females exhibited a wide and long sulcus. Similarly, Novak et al. also reported the sexually dimorphic trait in this area: preauricular was absent in males (94.8 %) and present in females (63.4 %) of European and African descent. The posterior sulcus was present in 27 percent of males and 85 percent of females. ^{Cardoso & Saunders (2008}) tested the intra-inter observer errors of the ilium for sex classification, and results showed that the agreements were adequate (0.67-0.89 and 0.50-0.76 for intra- and inter-observer agreements, respectively).

The goal of this study is to increase the reliability of utilizing posterior ilium for sex estimation. For this reason, we applied the convolutional neuralnetwork to this study. The number of researches using deep learning approach has increased rapidly in the medical imaging field for detection and diagnosis, such as MRI, microscopy, CT, ultrasound, and X-ray (^{Litjens et al., 2017}). Deep learning is a subset of machine learning that constructs computational models, inspired by the structure and function of the human brain, called artificial neural networks. Deep learning is used to perform various tasks, such as speech recognition, visual object recognition, and object detection (^{LeCun et al., 2015}). One type of deep learning neural network is a convolutional neural network (CNN), which shows promising applications in medical image analysis, classification of images for the diagnosis, localization for anatomy education, detection of the lesion or tumor, registration, and segmentation of images for focusing on the important features (^{Ker et al., 2018}). In recent studies, the high performance pre-trained networks of CNNs were developed, including AlexNet, VGGNet, GoogLeNet, ResNet, and ResNet. AlexNet and GoogLeNet are generally used for image classification and feature extraction with good results (^{Alaskar et al., 2019}).

Recently, GoogLeNet (also called Inception-V1) was developed and excelled in object recognition as well as classification. GoogLeNet consists of main four layers: a dropout layer, a fully connected layer, a softmax layer, and a classification-output layer, for a total of 22 layers. The concept of GoogLeNet is combining multiple-scale convolutional transformations by employing merge, transform, and split functions for feature extraction (Alaskar et al.; ^{Alzubaidi et al., 2021}).

The experiments were designed to classify the sex of an individual by visually assessing the posterior portion of the ilium from dry bones and os coxa images, which include following features: the preauricular sulcus, postauricular sulcus, the elevation of the auricular surface, and the iliac tuberosity. In addition, GoogLeNet approach was also developed for this study. The purpose of this study was two-fold. Firstly, this study aimed to compare the accuracy of the visual assessment methods in dry bones and 2D images. Secondly, this study also aimed to test the accuracy of the convolutional neural network (GoogLeNet) to improve the performance of a sex determination tool in the Thai population.

MATERIAL AND METHOD

The study protocol was approved by the Research Ethics Committee, Faculty of Medicine, Chiang Mai University, Thailand (Research ID: ANA-2563-07285). The os coxae were provided by the Forensic Osteology Research Center (FORC), Faculty of Medicine, Chiang Mai University. The total samples consisted of 250 bones that were divided into 200 as a ‘training’ group (100 females, 100 males) and 50 as a ‘test’ group. The age at death ranged from 26 to 94 years. The selected os coxae for this study were all left-sided, complete, and from the Thai population. Subjects were excluded from the study based on the following: fractured or incomplete os coxa, a sign of pathology, from a non-Thai person, and with the age less tan 20 years or over 100 years.

The visual assessment focused on the auricular area of the pelvis. The considered parameters for sex determination scoring system for each individual using dry bones and 2D images included preauricular sulcus, postauricular sulcus, elevation of superior, inferior, ventral and dorsal edges, micro-porosity, macro-porosity, and iliac tuberosity (Table I). The preauricular sulcus was recorded as an absence if it was not found (Fig. 1A), a streak if separation was in process (Fig. 1B) and a sulcus if it was completely separate (Fig. 1C) (^{Mahakkanukrauh et al., 2012}). The postauricular sulcus was recorded as absent (Fig. 1D) and the presence of the sulcus (Fig. 1E) (Mahakkanukrauh et al.). The elevation was observed at four edges of the auricular surface, including superior, inferior, ventral, and dorsal (Fig. 2A) (Wescott). Each edge was given a score individually as 0 if completely elevated (Fig. 2B) and as 1 if partially or non-elevated (Fig. 2C). On the auricular surface, the porosity of this area was observed. Microporosity was recorded if a diameter was less than 0.5 mm (Fig. 3A), and macro-porosity was recorded if diameter was greater than 0.5 mm. (Fig. 3B). The score was given 1 for the presence and 0 for the absence of the micro- and macro-porosities, which were recorded individually. The iliac tuberosity was recorded as 0 if the surface was smooth (Fig. 3C) and 1 if the surface was rough (Fig. 3D). The intra- and inter-observer reliability tests were assessed in both dry bones and images by the first and second observers. The 50 samples were randomly selected from 200 training samples.

Table I Morphological traits and scoring system for the visual assessment. 

Fig. 1 The variation of preauricular sulcus: absent (A), streak (B) and sulcus (C). The variation of postauricular sulcus: absent (D) and sulcus (E). 

Fig. 2 Four edges of the auricular surface include superior, inferior, ventral and dorsal (A), the ventral edge is completely elevated (B), and partially or non-elevated (C). 

Fig. 3 Micro-porosity with a diameter less than 0.5 mm (A), macro-porosity with a diameter greater than 0.5 mm (B), Texture of iliac tubercle appeared smooth (C), and rough (D). 

Technical photography. This study used 2D images of the os coxae. All images of the bones were taken with a digital camera with technical specs: Sony a57 Lens; Sony dt 18 - 55mm F3.5-5.6 SAM, at focus on 55 mm, autofocus mode iso 200. The pictures were saved in ARW file format. Photographs were taken using the top view and the distance between lens and bones was standardized in every photograph. Each left os coxa was placed on a black silk velvet background. Each bone was positioned with the auricular surface of the ilium facing upwards with the pubic symphysis pointing to the top. The camera lens was parallel to the os coxa. The anatomical landmarks of the os coxa marked on the camera screen grid were the auricular area. This position was in the same plane for all images.

Segmentation. In deep learning method, the 200 samples were derived from the image-based as well as dry-bone groups. The os coxa images contain all the parts of the pelvis. As the auricular area was the focused area in this study, the segmentation was performed to determine the region of interest (ROI). All images were cropped by Ado- be Photoshop 2020 software, which comprised the features of auricular area, such as preauricular sulcus, auricular surface, postauricular sulcus, postauricular area, and iliac tuberosity (Fig.4).

Convolutional neural networks (CNN). The pre-trained GoogLeNet neural network, a convolutional neural network (CNN), consisted of 22 layers, was used for the training process. Initially, the 200 cropped images of the training set, sized 224 x 224 pixels, were input into the Deep Network Designer application in MATLAB. The Fully Connected Layer and the final class Output Layer were adjusted for this process. In GoogLeNet, these layers were named ‘loss3- classifier’ and ‘output classification layer’, respectively. The output size number of the new fully connected layer was changed from 10 to 2. The data was then imported, and the augmentation options adjusted to consist of random rotation from (-30˚) to 30˚and random rescaling from 0.9 to 1.1. The validation set was split from the training data at 30 percent (60 samples). For training options, adjustments to the hyperparameters were carried out for the highest accuracy, such as tuning the initial Learning Rate, validation frequency, MiniBatchSize, MaxEpochs, L2Regularization, and Momentum. Validation accuracy (%) was derived after the training. Results were exported and tested in the test group for sex classification processing (Fig. 4). The deep learning process was developed on Notebook Lenovo IdeaPad Gaming3 15IMH05 81Y400PATA with 512 GB of memory and 4GB of GDDR6 GPU memory (NVIDIA GeForce GTX 1650 Ti). A convolutional neural network (CNN), the GoogLeNet ran on MATLAB 2020a.

Statistical analysis. Descriptive statistics were used to des- cribe the variables. For visual assessment, the data was analyzed by using discriminant statistics for sex classification. After the discriminant analysis, the sex discriminant function was formulated from SPSS and was tested on 50 testing samples. The intra- and inter-observer reliabilities were assessed using Cohen's kappa coefficient (^{Mukaka, 2012}). IBM SPSS statistics version 22 was used to examine the data.

Fig. 4 Deep learning algorithm framework for sex prediction in both training and test sets. We selected ROIs of the os coxa images and used the GoogLeNet to extract features and then predict the sex. 

RESULTS

Visual assessment of dry bones. The descriptive statistics results showed that the postauricular sulcus and dorsal edge elevation variables showed differences between males and females. The postauricular sulcus was found in 69 percent of females and was absent in 96 percent of males. Dorsal edge elevation was detected in 61 percent of females and was absent in 86 percent of males. When the 200 training samples were analyzed, the highest accuracy was 95 percent when using all variables, and the accuracies were 94 percent and 76 percent for males and females, respectively. For the test samples, the accuracy was 72 percent. The intra- and inter-observer reliabilities showed a moderate agreement for the scoring method (Kappa = 0.54, 0.67).

Visual assessment of images. The descriptive statistical results showed that the preauricular sulcus, postauricular sulcus, and dorsal edge elevation variables were sexually dimorphic. The preauricular sulcus was found in 60 % of females and absent in 63 % of males. The postauricular sulcus was found in 78 percent of females and absent in 67 % of males. The dorsal edge elevation was found in 62 % of females and absent in 83 % of males. The highest validation accuracy was 79.5 % when preauricular sulcus, postauricular sulcus and dorsal edge elevation were used, and 78.5 % accuracy was achieved when all variables were utilized. The accuracies for males and females were 87 % and 72 %, respectively. The accuracy was 60 % for the test samples. The intra- and inter-observer reliabilities showed a moderate agreement for the scoring method (Kappa = 0.61 and 0.60).

Deep Learning method. The best hyperparameters of this experiment were the initial Learning Rate = 0.001, validation frequency = 25, MiniBatchSize = 8, MaxEpochs = 50, L2 Regularization = 0.0002 and Momentum = 0.9. The training took a total of 7 minutes and the accuracies were 93.33 % and 88 % for the validation set and test set, respectively.

DISCUSSION

Our study found the high-performance parameters that differed between males and females in the visual assessment of both dry bone and images were postauricular sulcus, preauricular sulcus, and dorsal edge elevation. The postauricular sulcus was found in 69 percent of females and absent in 96 percent of males for the dry bone visual assessment. In the image-based assessment, it was found in 78 percent of females but was not found in 67 percent of males. The streak was observed in 15 percent of the preauricular sulcus in all samples. The preauricular sulcus was found in 78 percent of females but was not found in 29 percent of males. In comparison, when we observed in the images, we found that the streak was present in 8 percent of the preauricular sulcus in all samples. The preauricular sulcus was found in 60 percent of females and absent in 63 percent of males. The dorsal edge elevation was noted in females and nonexistent in males, 61 percent and 86 percent, respectively, among dry bone samples. In the image-based observation, the dorsal edge was elevated 62 percent in females and non-elevated 83 percent in males. The accuracies of visual assessment were 85 percent and 79.5 percent for dry bone and 2D images, respectively. The results showed that the accuracies of the two methods were dissimilar, despite using the same samples. The dry-bone method had higher accuracy than 2D images due to image limitations, such as low image quality and the difficulty of observing the streak or sulcus in the images.

Our finding of the posterior auricular sulcus in the dry-bone visual assessment was consistent with Mahakkanukrauh et al. which studied the same Thai population. The results of the two studies showed the almost presence of sulcus in 60 - 69 percent of females and almost absence in 94 - 100 percent of males. Similarly, Wescott et al., which studied the samples of European descent and Mahadevappa & Shivalingaiah, which investigated the samples of Indian descent, found the sulcus in females but mostly absent in males. However, ^{Gohil et al. (2014}) reported that the posterior auricular sulcus was the most efficient parameter for sex discrimination in females (72.22 %) but ineffective in males (29.62 %) in the Indian populations, similar to Mahadevappa & Shivalingaiah. Consequently, the sexually dimorphic characteristic of the postauricular sulcus was mostly found in females and mostly absent in males.

Our findings of the preauricular sulcus were similar to those of Mahakkanukrauh et al. that recorded the frequencies of streaks, sulci, and its absence in the Thai population. The results showed that the streak and sulcus were mostly found in females and notably absent in males. Consistently, Gohil et al. reported that the preauricular sulcus that was scored as either less marked or more marked, the results showed more marked in 68.51 percent of females and less marked or absent in 48.14 percent of males. Additionally, the preauricular sulcus was also assessed on a 5-point scoring system by Novak et al., in European and African samples, and Karsten, in American Blacks and whites. Their results similarly indicated that the trait was present in females and absent in males. Morphologically, male sulci are displayed as short and narrow whereas wide and long among female sulci.

The elevation of the auricular surface was observed in this study. We divided the surface into 4 edges, according to Wescott. Our results showed that only dorsal edge was sexually dimorphic. It was elevated in females (61 %) and not elevated in males (86 %). In both sexes, the auricular surface was partially elevated, whereas Wescott found no or partial elevation in males (99.4 %) and complete elevation in females (66.7 %). Similar to Wescott’s findings, Novak et al. also found that the elevation was either non-existent or just partially present in males (84.5 %) and entirely present in females (74.3 %). However, this sex indicator was not a useful parameter in Thais.

In our study, the intra- and inter-observer correlations of visual assessment methods showed a moderate agreement (Kappa = 0.54 - 0.67) (Gohil et al.). Similarly, Cardoso & Saunders found that intra- and inter-observer agreements were both adequate, 0.67 - 0.89 and 0.50 - 0.76, respectively. As a result, the visual assessment methods, both dry bone and image-based, were deemed more subjective, so the convolutional neural network (CNN) approach was employed to eliminate subjectivity in terms of determining the sexes from the auricular area.

We improved the efficiency of the tool for classifying sex by using CNNs. In our study, the deep neural network method demonstrated higher accuracy than the visual assessment methods in both validation and test samples. The GoogLeNet is one type of convolutional neural networks (CNNs) that excels in image classification system (Alaskar et al.). In many studies, GoogLeNet was mostly used for image classification. ^{Takiyama et al. (2018}) used the deep convolutional neural network (GoogLeNet) and CNN system to classify the stomach EGD images (the total of 27,335 images) with high accuracy (97 %). Furthermore, ^{Kovalev et al. (2017}) compared the conventional and deep learning methods of classification of chest radiographic images. They found that the deep learning had a higher accuracy than the conventional method, and GoogLeNet performed slightly better than AlexNet CNN. Furthermore, ^{Liu et al. (2020}) and ^{Dong et al. (2017}) compared various deep neural networks for classifying medical images, including AlexNet, GoogLeNet, ResNet, VGG16, and LeNet-5. Both studies reported that GoogLeNet had the best performance in image classification. Based on our findings and the previous studies, GoogLeNet presented a useful application for image classification. The limitations of our study should also be addressed. The samples were too insufficient (n = 200) for the efficient deep learning training in the current study, compared to other studies which utilized more than 2000 images (Dong et al.; Kovalev et al.; Takiyama et al.; Liu et al.). Thus, the CNN training with more samples could lead to higher accuracy. Lastly, the deep learning technology can also be applied in other non-medical fields for image classification. Our deep-learning tool performed relatively stronger in classifying sexes based on auricular area images compared to the visual assessment of dry bone. This finding indicates that deep learning can be an effective sex estimation method in Thais.