Determining the Optimum Maturity of Maize Using Computational Intelligence Techniques

In recent times, the phenomenal increase in the population of people and livestock in the world has placed an enormous pressure on water and land resources used by both crop farmers and herders alike. Desertification, deforestation and uncertainties in climatic conditions in Sub-Saharan Africa have led to massive movements of herders in search of pasture with resultant conflicts with local farm communities in the region. The inability to find a lasting solution to these problems has led to persistent cases of deteriorating relationships amongst crop farmers and herders which has continued to precipitate hostile consequences including the loss of lives, interruption and annihilation of the family units and in some cases, whole communities are destroyed. This research attempts to address the problem of inadequate grazing resources by the use of advances in Computational Intelligence Techniques in the determination of the optimum maturity of maize, so as to complement for the grazing of livestock in the region. Although the challenge inherent in determining the optimum maturity of maize is by no means trivial, the practice was hitherto based on human perception, which is a function of experience over time. This paper leverages on the use of Artificial Neural Networks (ANN) interfaced with image processing and Convolutional Neural Networks (pre-trained ResNet50 Network) in determining the optimum ripeness of the maize crop grown in Sub-Saharan Africa. Results obtained indicated a 3.5% improvement classification accuracy of pre-trained ResNet50 over ANN model, providing a stimulus for further research on the subject area. Therefore, this research posits that farmers could be sensitized on the possibility of utilizing image processing and neural networks technique in the determination of the maturity of maize in the nearest future when made operational.


Introduction
The climatic conditions across Sub-Saharan Africa support the cultivation of various agricultural products. Farmers of agricultural crops are usually natives of such areas or regions.
Livestock farmers or herders usually migrate from the arid regions to the rain forests within Sub-Saharan Africa in search of pastures for their livestock.
In recent times, a phenomenal increase in the population of people and livestock in the region has placed enormous pressure on water and land resources for both the farmers and herders. Desertification, deforestation and changes or uncertainties in climatic conditions have also led to massive movements of herders in search of pasture resulting to conflicts with local farm communities in some parts of the region such as in Nigeria [1]. The above has also led to a decline in meat and dairy products.
Establishment of grazing reserves has been suggested by experts which could mitigate the conflicts. This has been met with resistance and more conflicts, and it encourages little pastoralism which may not solve the problem in the long run.
The use of security personnel, especially the military is another measure to contend the issue. This measure is said to be ineffective in solving intra-state conflicts in Sub-Sahara Africa. Fragmentary sectional methods to the conflict situation and livestock development will not serve either.
The inability to find a lasting solution to these problems has led to deteriorating relationships amongst farmers and herders.
Hence, there have been hostilities leading to loss of lives, interruption and annihilation of the family unit, and in some cases, communities are destroyed [2].
The global increase in human and livestock population has necessitated the increase in food production. Maize is one of the many agricultural crops that can be cultivated under diverse climatic conditions in Sub-Sahara Africa for its carbohydrate, fat and protein contents, besides wheat and rice. In developed countries, maize is commonly used as a secondary source of food, unlike in Sub-Sahara Africa, where it is a staple food [3]. In this regard, maize is consumed in different ways, namely: roasted, cooked and refined into flour or porridge.
Amongst crops cultivated in Sub-Sahara Africa, maize (which is cultivated under diverse weather conditions) ranks highest due to its variety of uses and nutritional contents. The entire maize plant is a good source of food for livestock. Nevertheless, the plant is often allowed to attain biological maturity in Sub-Sahara Africa, where its water and nutritional contents are wasted. Before biological maturity, the maize kernel is ready for harvest, and the maize plant still retains its greenish coloration to some degree.
Besides its traditional usage, the need to consider the maize plant after harvest as an additional source of grazing for livestock in Sub-Sahara Africa could always be beneficial. This is due to its high energy and water contents, and the fact that maize can be cultivated all-year round. Some of its varieties have a high content of beta-Carotene for vitamin A and as one of the major staple crops that characterise the green revolution in the developed world as presented in [4,5]. Maize requires more energy compared to most grain crops during photosynthesis, and the yield of its kernel per hectre tends to be more than most grain crops as stated by Plessis in [6] and reported by Peter et al. in [1].
Maize plant contains more foliage than most grain crops ranging from as low as eight leaves to twenty leaves, occurring in an alternating fashion to provide some degree of stability to the plant and equally absorb sunlight from any direction for photosynthesis. Maize leaf has the ability to absorb water quickly during moist conditions and loses the same during adverse conditions. However, the leaves maintain a mechanism of bending inward to lessen the excessive loss of water during drought and scourging sun [1].
Before biological maturity, maize leaves contain a substantial amount of nutrients. The stems contain a significant amount of water and are still fresh, which can be used to feed livestock. Hence, there is a need for farmers to harvest maize before biological maturity so that livestock graze on the stalk and leaves.
The readiness of the maize plant for harvest is achievable between the hard dough stage and the physiological maturity period. In determining the appropriate time for the readiness of maize for harvest, the human eye is not very dependable due to differences in vision by individuals [7]. The need to use Image Processing (IP) method that does not suffer from the problem of weariness experienced by human eyes and other related problems associated with the human eye like poor vision and differences in perception when determining the readiness of maize for harvesting at the physiological stage becomes eminent.
Images acquired through the method of IP must be classified using a standard and reliable approach which could be Supervised Learning Method or Unsupervised Learning Technique. The Supervised Learning Approach is adopted in this research since the pre-classification of images is essential. Artificial Neural Networks (ANN) and Convolutional Neural Networks (CNN) are implemented under a supervised learning condition due to the universal capability of the Neural Networks to approximate any nonlinear function.
The researchers in [1,8] attempt to use Computational Intelligence Technique in determining the optimum maturity of maize by its leaves. However, maize leaf images cannot be relied upon since they do not play a significant role in determining the optimum maturity of its kernel. This is because the leaf's green coloration is a function of the nutrients obtained by the maize plant as well as photosynthesis. Also, there is no mention of the variety of maize used in this research; varieties of maize exist and are continually modified genetically to suit different climatic conditions [9]. Besides, the methodology applied to capture maize leaf images does not specify the distance and angle at which the camera captures the sixty images used in the classification. This provides information on the resolution and precision of the images captured. In addition, the results obtained show that the networks are able to achieve an average accuracies of 80% and 100% respectively, which is quite impressive. However, the best validation performance graph for the 80% showed that there are serious over-fitting and under-fitting on the training, validation and test data. The best fit line is not gotten by the training, validation and test data set (see Figure 1a). While in the case of 100% classification accuracy, the best fit line is clearly missed (see Figure 1b). This clearly indicates that the classification has a serious problem of over-fitting though with 100% classification accuracy.  Finally, in an effort to differentiate the maturity of maize, maize crops are planted on different pieces of land and at different times. These variations in farmland and planting period result in the images acquired from different sources which cannot give reliable results.
In this paper, a comprehensive framework for determining the optimum maturity of maize using Computational Intelligence (CI) tools and image processing techniques is proposed.
Determining the optimum ripening of maize through the application of Computational Intelligence (CI) tools and image processing techniques allow for the development of an integrated crop-livestock system that is beneficial to both farmers and herders. CI is an embodiment of tools such as NNs, Fuzzy Logic (FL), Probabilistic Reasoning (PR), Belief Networks (BN), Genetic Algorithms (GA), Evolutionary Programming (EP) and Artificial Life (AL) as reported in [10].
Reference [11] states that a CI system handles merely numerical data, possess components that are capable of recognising patterns, adapt to changes, tolerates faults, utilises human speed in the completion of any task, and error reduction competence. In this work, Artificial Neural Networks (ANNs) and Convolutional Neural Networks (CNNs) have been deployed for the purpose of classification as they possess most of the attributes of a CI system mentioned above.
The rest of the paper is organised as follows: Section 2 presents the review of related works; Section 3 presents the detail of the methodology, while Section 4 narrates the details of the results and discussions. The paper is concluded in section 5 with a summary of the research findings pointing the way for future research on the subject area.

Review of Related Works
To improve the modeling supremacy of NNs, ANNs which simulates biological neurons in a restricted way have been extended to CNNs. CNNs are the utmost representation of supervised learning models which apply filter or mask kernel or template window on each input for the purpose of classifications. This section reviews related works in the detection and or classification of agricultural products using Computational Intelligence /Machine Learning tools.
In recent times, much work has been conducted in the classification of agricultural products using Machine Learning Techniques. In [12], Yulcin and Razavi report the use of CNN in the classification of 16 plant species of cherries, beans, pomegranates, apricot, and so on using camera images of 1,200 agricultural stations in Turkey and obtain 97.47% accuracy. In the same vein, CNN was applied in the detection and classification of about 6000 leaf images obtained from a natural environment. Over 5000 leaf images are used as training set and about 1000 leaf images for validation purposes. The leaves are categorised into four labels: bacterial spot, yellow leaf curl, late blight and healthy leaves.
Classification accuracy of 99.32% is reported in [13]. In [14], Prashar et al. report that cotton farmers in India lose about 10%-20% annual profit due to diseases that affect the crop. They use the Overlapping Pooling Technique in CNN with Multilayer Perceptron (MLP) in detecting infected and healthy leaves of the American cotton leaf disease in order to mitigate the problem of profit loss in the cotton industry. A 96% detection accuracy is achieved by the CNN model.
In [15], Boulent et al. use three types of data sets depending on their complexity in the automatic identification of plant diseases: The first image samples are captured under controlled situations where the leaves are placed on the same background and brightness. The second image samples are taken under unrestrained conditions and focus is made on specific plant organ. The last image samples are taken in unrestrained situations too but there is no special emphasis on any plant organ. The paper report that in research work relating to detection and classification of crop diseases, CNN generates a supreme performance and can handle complex situations relating to image processing and classification. The potentials of CNN can equally make them to be part of operational tools, even though certain measures need to be taken into consideration.
In [16], Jiang et al. propose a novel detection approach that uses a deep-CNNs derived from Single-Short multi-box Detector (SSD), GoogleNet Inception module in order to enhanced the extraction potential. Finally, Rainbow hybridisation approach in R-SSD is incorporated to enhance the feature fusion performance by using several angle feature fusions. The improved model called INAR-SSD generate an average detection precision of 78.80% when applied on the real time detection of five apple leaf diseases viz: Rust, Mosaic, Alternaria leaf spot, Grey spot and Brown spot.
In [17], Lin et al. propose a matrix-based CNN (M-bCNN) that solves the problem of disappearing gradient and over-fiting of data; that is achieved by the parallel arrangement of the convolutional layers in matrix form and the incorporation of DropConnect, Local response normalisation, exponential proportionate unit, and so on. The experimental samples are 16652 images comprising of 8 categories. The images are later augmented to 83260 images. They report a 96.5% validation and 90.1% testing accuracies in the image classification of fine-grained wheat leaf diseases. Hence, they claim that M-bCNN outperforms Alexnet and VGG-16 pre-trained networks.
In order to detect wheat mite, a common and dangerous pest that affects wheat growth, in [18], Li  Saleem et al. in [21] state that CNN possesses great potentials for improved accuracy in the detection and classification of plant diseases.
To achieve automation in harvest management, in [22], Gonzalez et al. report the use of a network that depends on Mask R-CNN in the detection of blueberry images. ResNet50, ResNet101 and MobileNetv1 are equally verified on the data. ResNet50 give the best outcome of 0.759 mean average precision of 0.759 on a threshold of 0.5 of intersection over union error.
K-Means Clustering algorithm (FCM-KM) and Faster R-CNN Fusion are applied on 3010 images of rice for the rapid detection of rice blast, bacterial blight and sheath blight diseases. The result shows a disease detection accuracy of 96.71% for rice blast, 97.55% for bacterial blight and 98.26% for sheath blight of rice as reported in [23] by Zhou et al.
In [24], Elhassouny and Smarandache report an inspiration from MobileNet CNN to recognise 10 common types of Tomato leaf diseases. Their results show that the lower the learning rate the more the accuracy of recognition of the diseases.
Li et al. in [25] develop a technique for Soya beans counting in order to improve crop breeding. The technique is formed from the combination of density approximation and CNN techniques. The results obtained show that the approach would be promising on precision task of future breeding.
Convolution Neural Networks (CNNs) and Gated Recurrent Unit Networks (GRU) are implemented in distinguishing unclear crop differences. Results obtained from several temporal data show that the approach obtained the best performance, particularly on the classes with common phonological rules as reported in [26] by Li et al.
The average of 99.58% prediction accuracy for optimising maize yield in a plantation is achieved by a CNN model developed for image recognition in [27] by Abdullahi et al.
Li et al. in [28] enhance the performance of CNN by changing the number of Convolution layers and the Pooling layers in models such as VGG, AlexNet, and CaffeNet in the classification of distant sensing scene and their results confirmed that the number of layers are directly proportional to the accuracy of the model as reported by Sinha et al. and Haryanto et al. in [29] and [30] respectively. In the same vein, in [31], Zhang et al. determine the accuracy and scalability of Tomato ripeness employing the use of CNN in the design of Tomato reaping robot for classification and prediction. Their results show a 91.9% and <0.01s accuracy and prediction time respectively. They equally recommend the use of CNN in related research areas.
In the segmentation and classification of several organs in the medical discipline, Harouni et al. in [32] proposed a universal network model based on Unet architecture of CNN that has the potential of segmenting unalike organs in diverse modalities, resolve segmentation and classification issues concurrently. Their work report 99% accuracy in classification and a mean dice score of 89% in the accuracy of the segmentation.
In [33], Xia et al. report that ANN outperforms Support Vector Machine (SVM) in the hyperspectral identification and classification of two Rape varieties of oilseeds: Ning You 22 and NingZa 19. In like manner, Hu et al. in [34] reported that though Time Series images with the potential to describe periodical features of agricultural crops due to their great progressive tenacity do not essentially generate better classification accuracy. Other application of CNN can be seen in [35] as reported by Do et al. as it is used for face and non-face classification.
In summary, it can be seen that NN has a high promising potential in the classification of agricultural crops as well as other objects, hence its application. This work is an extension and improvement on the work reported by Peter et al. in [1] and Peter and Abdulkadir in [8]. Moreover, to the best of our knowledge no one has done a work of this nature.

Methodology
In this section, a description of the basic materials and techniques applied in the research are presented. Prime among the materials are the maize variety cultivated, the climatic condition, the selection of farming site, land preparation, the maize comb images collected and the image acquisition tools. Furthermore, the techniques used for the image processing as well as its classification are also presented in this section.

Maize Variety
The advancement in science, specifically crop science has led to the improvement of the classical maize varieties into different potent species suitable for different climatic conditions. In this research, SAMAZ 17 maize seedling variety that is highly resistant to drought, striga conditions and contain a good amount of protein is used for cultivation of the farm land [9,36].

Climatic Condition
The maize variety SAMAZ 17 is cultivated under a tropical climate with an average daily temperature of 83°F in the months of June to October. The rain fall has a probability greater than 0.46 inches within the period of June to October as reported in [37] by Gelaro et al.

Farming Site Selection and Preparation
The site for the cultivation of SAMA17 is a three-hectre farm land having very good drainage. Glyphosate is applied to destroy weeds and the land is left for two weeks before plowing, harrowing and ridging to ensure that glyphosate does not affect the maize seeds.

Pre-emergence
In the second day after planting the maize seed (SAMA17), 2.5 litres of Paraquat and 5 powder sachets of Combi plus are diluted in 1 litre of water and the solution applied per hectare to ensure that those weed seeds that have been disperse by the wind into the farm land after the 2 weeks of the application of glyphosate has expired are destroyed.

Post-emergence
After the emergence or germination of the maize seeds planted, 2 litres of guard force (selective herbicide for maize) and a sachet of guard force are applied per hectare as herbicide. Two weeks after the planting, 3 bags of NPK are applied per hectare. In the fifth week, 3 bags of NPK are applied again. Finally, at the eighth week 2 bags of Urea are applied to the maize farm per hectre.

Image Collection
Over one thousand maize comb images (see Figure 2a) are first collected at the tenth week using a digital camera FINEPIX Z35 FUJIFILM Full High Definition Charge Couple Device (CCD) at 90° and about 3 inches distant from the maize comb with its flash light on. These particular maize plants are marked to ensure consistency in the samples that are later collected. In the eleventh week, another set of over one thousand maize comb images (see Figure 2b) are collected. Furthermore, finally, in the twelfth week, another set of over one thousand comb images (see Figure 2c) are collected using the same approach. The tenth, eleventh and twelfth weeks' period represents the different maturity periods of soft dough stage, hard dough stage and the optimum ripening stage respectively.

Hardware Software Experimental Setup
The experiment is performed on Windows7 Ultimate 64-bit Operating System Service Pack1. The processor is Intel® Core (TM) i7, 2620M CPU @2.70GHz. The RAM is 8.00GB and HDD is 500GB. The MATLAB scripts and Toolbox are implemented on MATLABR2018a.

Image Processing
The images acquired are replicated into two parts, for use in training ANN model and CNN pre-trained Restnet-50 Network.

Image Processing for ANN
In an effort to enhance the focusing of the region of interest on the images, Paint-Brush software is used in rescaling the images to a sizeable dimension. A MATLAB script is developed and used to convert images from Red, Green and Blue (RGB) to Hue, Saturated and Value (Intensity) (HSV) colour models respectively. The individual components of HSV is extracted and then randomly de-noised to maintain its key features.
In order to improve the image quality, the value component of HSV colour model (i.e. the intensity) of each image is sharpened to preserve the originality of the image. The sharpened images are later combined with the hue and saturated components of the respective images in order to realise the target objectives In maintaining the precision and identification of images, colour and texture characteristics are applied for the purpose of classification. For the colour feature, mean and standard deviation of the HSV colour model are used after converting the RGB colour model to HSV colour model. In the case of the texture feature, Grey Level Co-occurrence Matrix (GLCM) is used to create a gray-level co-occurrence matrix from the images. The standard deviation of the statistical properties (Contrast, Correlation, Energy and Homogeneity) of the GLCM are obtained. The features used in ANN are the mean and standard deviation of the HSV colour model which comprises of 6 (six) features and the standard deviation of Contrast, Correlation, Energy and Homogeneity being the statistical properties of the GLCM. There are a total of 10 (ten) features (elements) of 3000 data used as input

Image Processing for ResNet50
A MATLAB script is developed and used for the image processing and classification scheme.
Pre-trained Restnet-50 Network takes input size of [224, 224, 3] which means image dimension of 224 X 224, and 3 channel images. The gray scale images are converted to RGB colour model to suit the channels.
Each layer of the CNN produces a response. However, not all of the layers are suitable for image extraction. Layers at the beginning captures basic image features such as edges and blurred.
In order to improve the images for the CNN, the weights for the second convolutional layer are converted from matrix to gray image. The features extracted are processed by deeper network layers which combine the earlier features to form higher level image features. These higher level features are better suited for image recognition task because they combined all the primitive features into richer image representations. Additional features are extracted from one of the deeper layers, in particular the fully connected layer 1000 before the classification layer which is referred to as "fc1000" in pre-trained Resnet-50 Network by using activation methods.

Classification Scheme for ANN
The prediction of the inputs class is based on 3000 samples of 10 elements and the targets class based on 3000 samples of 3 elements. The 3000 samples are divided into 3 parts of 2100 samples (70%) for training, 450 samples (15%) each for validation and testing respectively.
The training function used is scaled conjugate gradient backpropagation due to its suitability in low memory situations. The number of hidden layers are 35 hidden layers.

Classification Scheme for CNN
The input class are 3000 samples randomly divided into training features of 1000×2100 being 70% and randomly divided into test features of 1000×900 being 30%.
The labels of the training set are trained using the function called Fit-Class Error Correcting Output Codes (FitCECOC). The function returns a full trained multiclass error correcting output coding model. The function uses ( 1) / 2 K K − binary SVM models. K = # unique class labels.
The coding design is one versus all coding and the learner is a linear classification.

Results and Discussion
This section presents the results of the research findings. Figure 3 is the all confusion matrix of the results obtained using the ANN model for classification. The average classification accuracy for the ANN model is 64.5%. This result seems to be far lower than that obtained in [1,8].     Figure 5 is preferable than that reported by the authors in [1,8]. In addition, the average accuracy of 64.5% obtained is due to the similarity index of the data samples used in both the training, validation and testing sets are the same, except that the images are captured at different time intervals.
In Figure 6 the Receiving Operating Characteristics (ROC) tends towards the true positive rate which is an indication that the classifier is not only effective but efficient. Figure 4 presents 68% classification accuracy recorded amongst others from the script developed in MATLAB for pre-trained ResNet-50 Network. A 3.5% improvement over ANN model was achieved. This shows that the ANN model's prediction is quite valid considering that ResNet-50 has a fully connected 1000 layers (fc1000) before the classification layer based on the architecture of CNN which is expected to ordinarily outperform a classical NN containing only thirty-five (35) hidden layers before the output layer.

Conclusion
This paper addresses the challenges inherent in determining the optimum maturity of maize crops especially those grown in Sub-Sahara Africa. Two approaches are employed for the classification namely, ANN and CNN (pre-trained Network Resnet50). In ANN three datasets each comprising of a thousand data samples are obtained and preprocessed using image processing tools. Seventy percent (70%) of the sample was used to train the network; fifteen percent (15%) was used for validation and the remaining fifteen percent (15%) was used to test the ANN model. The resultant datasets serve as input to the ANN. The resulting classification result of the ANN model gave 64.5% average accuracy with a near perfect fit as seen in Figure 4. In Resnet50 the same three data sets are used except that there is no preprocessing of the sample images. Seventy percent (70%) of the sample was used to train the network; remaining thirty percent (30%) was used to test the pre-trained Resnet50 Network. The results obtained from Resnet50 show an improvement of 3.5% above the ANN model.
It is noteworthy that the results presented in this paper are based solely on the images of the maize comb acquired at different levels of maturity. Focus will be placed in the future on other parts of the maize plant in determining the maize optimum maturity.

Data Availability
The maize comb image data used to support the findings of this research are available from the corresponding author upon request.