Realizing Iris classification by fuzzy clustering -- computational intelligence operation (with MATLAB code)

Problem background

Iris flower dataset was created by Sir Ronald Fisher in 1936 [ 1 ] \color{#0000FF}{[1]} [1] The classical cube introduced can be used as a sample of discriminant analysis. The dataset contains three Iris species, iris setosa, iris versicolor and iris virginica, with 50 samples each. Each sample also has four characteristic parameters, namely, the length and width of sepals and the length and width of petals. Assuming that there is an iris plant now, how to infer which of the three categories it belongs to through its characteristics is a typical classification problem in machine learning.
Problem Description: try fuzzy clustering correlation method for analysis. When the number of clusters is 3, 4 or 5, the clustering results can be obtained and the corresponding graphics can be drawn.

Fuzzy clustering and fuzzy inference system

C-means fuzzy clustering

C-means fuzzy clustering (FCM) [ 2 ] \color{#0000FF}{[2]} [2] Is a clustering method that allows a piece of data to belong to two or more clusters. Among many fuzzy clustering algorithms, fuzzy C-means (FCM) algorithm is the most widely used and successful. It obtains the membership of each sample point to all class centers by optimizing the objective function, so as to determine the category of sample points, so as to achieve the purpose of automatic classification of sample data. It is based on the minimization of the following objective functions:

J m = ∑ i = 1 N ∑ j = 1 C u i j m ∥ x i − c j ∥ 2 , 0 ≤ m < ∞ {J_m} = \sum\limits_{i = 1}^N {\sum\limits_{j = 1}^C {u_{ij}^m} } {\left\| {{x_i} - {c_j}} \right\|^2},0 \le m < \infty Jm​=i=1∑N​j=1∑C​uijm​∥xi​−cj​∥2,0≤m<∞
m m m is any value greater than 1 1 The real number of 1, u i j u_{ij} uij# yes x i x_i xi. Membership in cluster j, x i x_i xi , yes i i i d d d-dimensional measurement data, c j c_j cj , is a cluster d d d-dimensional center is an arbitrary norm representing the similarity between any measured data and the center.
The above objective function is iteratively optimized to determine the membership degree u i j u_{ij} uij# and cluster center c j c_j cj # update and fuzzy division:

u i j = 1 ∑ k = 1 c ( ∥ x i − x j ∥ ∥ x i − c k ∥ ) 2 m − 1 {u_{ij}} = \frac{1}{{\sum\limits_{k = 1}^c {{{\left( {\frac{{\left\| {{x_i} - {x_j}} \right\|}}{{\left\| {{x_i} - {c_k}} \right\|}}} \right)}^{\frac{2}{{m - 1}}}}} }} uij​=k=1∑c​(∥xi​−ck​∥∥xi​−xj​∥​)m−12​1​

Takagi Sugeno fuzzy inference system

Takagi Sugeno Kang (TS) fuzzy reasoning [ 3 ] \color{#0000FF}{[3]} [3] , using a constant of the input value or a univariate output membership function of a linear function. Compared with Mamdani system, the deblurring process of Sugeno system is more effective in calculation, because it uses the weighted average or weighted sum of several data points instead of calculating the centroid of two-dimensional region. The final output of the system is the weighted average of all rule outputs:
F i n a l O u t p u t = ∑ i = 1 N w i z i ∑ i = 1 N w i {\rm{ Final Output }} = \frac{{\sum\limits_{i = 1}^N {{w_i}} {z_i}}}{{\sum\limits_{i = 1}^N {{w_i}} }} FinalOutput=i=1∑N​wi​i=1∑N​wi​zi​​

z i z_i zi is the rule output level, which is a constant value or linear function of the input value; w i w_i wi is the trigger strength of the rule; N N N is the number of rules. The following figure shows an input value of x x x and y y y's dual input system.

TS fuzzy reasoning has the characteristics of high computational efficiency, good use of optimization and adaptive technology, and is very suitable for mathematical analysis. Therefore, the classification task of this paper is to use FCM system based on TS fuzzy reasoning for analysis.

Experimental results and analysis

The experimental environment of this paper is MATLAB2018b. Firstly, preprocess the Irsi data set, disrupt the data set, and divide it into training set and test set according to the ratio of 8:2. The main idea is to use the FCM system based on TS fuzzy reasoning to generate the initial fuzzy clustering system, and then use the adaptive neuro fuzzy system [ 5 , 6 ] \color{#0000FF}{[5,6]} [5,6] to train and fine tune the fuzzy clustering system, and finally evaluate the accuracy and effectiveness of the system through the visualization results and mean square error.
The visualization results of cluster distribution can be obtained from the cluster center and membership matrix of the initial system, as shown in figures 3.1 and 3.2. From the figure, it can be seen that the third category can be well distinguished from the other two categories. The membership curves of the first and second categories still have large values at positions greater than 50 samples, and the distribution of sample points also overlap, so it is difficult to distinguish between the first and second categories.

After the initial system is trained and fine tuned by adaptive neural network technology, the error of FCM system based on TS fuzzy reasoning is reduced and the performance is slightly improved. The variation curves of training and testing errors with the number of iterations are shown in Figure 3.3 and Figure 3.4, and the mean square error is shown in Table 3.1.

Matlab source code

%%%computational intelligence  assignment,use FCM and anfis Cluster analysis of iris

iris = csvread('iris.csv');
iris = iris(randperm(size(iris, 1)), :);  %Disrupt data sets

[row, col] = size(iris);
iris_train = iris(1:uint8(row*0.8), :);  %Split dataset
iris_test = iris(uint8(row*0.8)+1:end, :);

setosaIndex = iris(:,5)==1;
versicolorIndex = iris(:,5)==2;
virginicaIndex = iris(:,5)==3;

setosa = iris(setosaIndex,:);
versicolor = iris(versicolorIndex,:);
virginica = iris(virginicaIndex,:);

Characteristics = {'sepal length','sepal width','petal length','petal width'};
pairs = [1 2; 1 3; 1 4; 2 3; 2 4; 3 4];  %Clustering with different "feature pairs"


Nc = 3;  %Number of cluster centers
M = 2.0; %Index of partition matrix
maxIter = 100;  %Maximum number of iterations
minImprove = 1e-5;  %Error condition of iteration stop
clusteringOptions = [M maxIter minImprove false];

%Initial configuration FIS parameter
genOpt = genfisOptions('FCMClustering', 'FISType','sugeno');  %use Sugeno reasoning
genOpt.NumClusters = Nc;  %Number of cluster centers
genOpt.Exponent = clusteringOptions(1); 
genOpt.MaxNumIteration = clusteringOptions(2);
genOpt.MinImprovement = clusteringOptions(3);
genOpt.Verbose = clusteringOptions(4);

inputData = iris_train(:,1:size(iris,2)-1);
outputData = iris_train(:,size(iris,2));
infis = genfis(inputData,outputData,genOpt);  %Define initial FCM fuzzy system 

%to configure anfis Training parameters
opt = anfisOptions('InitialFIS',infis);
opt.EpochNumber = 150;             % Maximum training rounds
opt.ErrorGoal = 1e-5;
opt.DisplayANFISInformation = 0;  % Print the information of fuzzy inference system
opt.DisplayErrorValues = 1;       % Print each epoch Training error
opt.DisplayStepSize = 0;
opt.DisplayFinalResults = 1;
opt.ValidationData = iris_test;  % Set validation data

[fis,trainError,stepSize,chkFIS,chkError] = anfis(iris_train,opt);  %train FIS,Return validation results

% Mean square error between output and true value of fuzzy system before training
trainout1 = evalfis(iris_train(:, 1:4), infis);
trainRMSE1 = norm(trainout1-iris_train(:, 5))/sqrt(length(trainout1))
testout1 = evalfis(iris_test(:, 1:4), infis);
testRMSE1 = norm(testout1-iris_test(:, 5))/sqrt(length(testout1))

% Mean square error between output and true value of fuzzy system after training
trainout2 = evalfis(iris_train(:, 1:4), fis);
trainRMSE2 = norm(trainout2-iris_train(:, 5))/sqrt(length(trainout2))
testout2 = evalfis(iris_test(:, 1:4), fis);
testRMSE2 = norm(testout2-iris_test(:, 5))/sqrt(length(testout2))

[minChkErr, n] = min(chkError);   % Returns the minimum error of the validation data and its corresponding training epoch number

s = ['\leftarrow The minimum error is', num2str(minChkErr),',epoch=', num2str(n)];
figure,
plot(trainError, 'linewidth', 1.5), hold on,
plot(chkError, 'linewidth', 2), 
xlabel('Number of iterations');
ylabel('Mean square error');
text(n, minChkErr, s, 'FontWeight','bold','FontSize', 14);
legend('Training error', 'Test error', 'Location', 'best');
set(gca,'FontSize',16);


%%%%%% Visualization of classification results %%%%%%%
[centers,U,obj_fcn] = fcm(iris,Nc,clusteringOptions);  
% % center:The clustering center obtained after iteration; U: Membership matrix; obj_fcn:The value of the objective function in the iterative process

%Draw the distribution of samples according to different "feature pairs"
figure('NumberTitle', 'off', 'Name','Sample distribution')
for i = 1:6
    x = pairs(i,1); 
    y = pairs(i,2);   
    subplot(2,3,i)
    plot([setosa(:,x) versicolor(:,x) virginica(:,x)],...
         [setosa(:,y) versicolor(:,y) virginica(:,y)], '.')
    xlabel(Characteristics{x})
    ylabel(Characteristics{y})
end
for i = 1:6
    subplot(2,3,i);
    for j = 1:Nc
        x = pairs(i,1);
        y = pairs(i,2);
        text(centers(j,x),centers(j,y),int2str(j),'FontWeight','bold','FontSize', 14);
    end
end

% Because the sample order is disturbed, the discrimination can be observed by arranging the membership values from large to small
U(1,:) = sort(U(1,:),'descend');
U(2,:) = sort(U(2,:), 'descend');
U(3,:) = sort(U(3,:), 'descend');
figure('NumberTitle', 'off', 'Name','Sample membership') ;
subplot(3,1,1);
plot(U(1,:),'-b');
title('Membership matrix value')
ylabel('first kind')
set(gca,'FontSize',16);
subplot(3,1,2);
plot(U(2,:),'-r');
ylabel('Class II')
set(gca,'FontSize',16);
subplot(3,1,3);
plot(U(3,:),'-g');
xlabel('Number of samples')
ylabel('Category III')
set(gca,'FontSize',16);

reference

[1] Fisher,R.A. "The use of multiple measurements in taxonomic problems" Annual Eugenics, 7, Part II, 179-188 (1936); also in "Contributions to Mathematical Statistics" (John Wiley, NY, 1950).
[2] Bezdec, J.C., Pattern Recognition with Fuzzy Objective Function Algorithms, Plenum Press, New York, 1981.
[3] Sugeno, M., Industrial applications of fuzzy control, Elsevier Science Pub. Co., 1985.
[4] Mamdani, E.H. and S. Assilian, "An experiment in linguistic synthesis with a fuzzy logic controller," International Journal of Man-Machine Studies, Vol. 7, No. 1, pp. 1-13, 1975.
[5] Jang, J.-S. R., "Fuzzy Modeling Using Generalized Neural Networks and Kalman Filter Algorithm," Proc. of the Ninth National Conf. on Artificial Intelligence (AAAI-91), pp. 762-767, July 1991.
[6] Jang, J.-S. R., "ANFIS: Adaptive-Network-based Fuzzy Inference Systems," IEEE Transactions on Systems, Man, and Cybernetics, Vol. 23, No. 3, pp. 665-685, May 1993.

Tags: MATLAB Machine Learning

Posted by A584537 on Sun, 08 May 2022 22:38:01 +0300