Библиографическое описание:

Олешко А. О. Hybrid K-Means Clustering Procedure Based on Radial Basis Function and Optimized by Artificial Fish Swarming Algorithm // Молодой ученый. — 2013. — №4. — С. 122-125.


Today swarm intelligence is a branch of artificial intelligence that studies the collective behavior and emergent properties of complex, self-organized, decentralized systems with social structure. Such systems consist of simple interacting agents organized in small societies (swarms). Although each agent has a very limited action space and there is no central control, the aggregated behavior of the whole swarm exhibits traits of intelligence, i.e., an ability to react to environmental changes and decision-making capacities.

Machine learning is the field of artificial intelligence that deals with algorithms that render computational models capable of learning and adapting to their environment. From an abstract viewpoint, machine learning is the procedure of extracting information in the form of patterns or rules from data. This purpose requires the use of computational methods. Human interaction can also be beneficial within a collaborative framework to the algorithms. However, the elimination of this necessity still remains the main challenge in the development of intelligent systems.

This material is dedicated to the conjunction of swarm intelligence and machine learning. In the following work is introduced a hybrid method for overcoming the problem of neural network training which consists in effective determining of the synaptic weights of a neural network to get the desired output for a set of input vectors.

Entry points

For investigation was chosen radial basis function network for its self-adapting and self-learning features. On the other hand it has several limitations with regard to convergence speed and forecasting accuracy. To overcome this problem artificial fish swarming algorithm inspired by the natural social behavior of fish in searching, swarming and following is used for optimization of radial basis function network.

The radial basis function network is a two-layer feed-forward network. Its hidden nodes implement a set of radial basis functions(e.g. Gaussian functions) and the training procedure is very fast. The input layer contains units of signal source, and the second layer is hidden layer. The number of units on the hidden layer is determined by necessity. The third layer is an output layer which reacts to input model. Movement from input layer to hidden layer is nonlinear and that from hidden layer to output layer is linear. Activation function of the units in hidden layer is RBF, which can be demonstrated by the following graph:

Fig 1. Structure of radial basis function networks

Hybrid K-means clustering procedure based on radial basis function and optimized by artificial fish swarming algorithm

An introduction step to clustering procedure is to train the model using original data. In order to determine an activation function center of each neuron in the hidden layer, width , weight from hidden layer to output layer the learning algorithms are used.

The learning algorithm can be divided into two parts. The rst part is clustering of all input samples and working out activation function center of units in hidden layer, and the width . The second part is to train the linking weights from hidden layer to output layer after the is determined.

Good robustness, global search ability, tolerance of parameter setting, and insensitiveness to initial values of AFSA(Artificial Fish Swarm Algorithm) can be used to upgrade the learning algorithms in determining parameters for neural network.

Assume the number of training samples is . First, initiate as a randomly generated training samples as center and width ,where , the number of hidden units. Then, according to the principle that training samples should be closest in distance to the center we adjust the center, and execute iteration until we find the right primary function . The next step is to work out the width with specific RBF (Radial Basis Function). Artificial fish swarm algorithm represents following solution steps:


Define the number of hidden units as the population of artificial fish colony; Training samples count is the maximum iterative times; Visual area of artificial fish is 1, moving step is . Using stochastic algorithm, we select different samples of number as initial center , where The number in brackets points to iterative number, i.e. — value of center at 0 iteration.

Define the initial vector of weights ; Each component vector is the random number between -1 and 1. Assume, the number of units in hidden layer is , and is the M-dimensional vector, the number 0 in the bracket means the first time of calculation. The acceptable network output error is In order to define the food concentration (FC) in the current position of each artificial fish we use an output error

as follows: represents the food concentration itself. The maximum value of food concentration will be included in the report and the value assignment of the individual fish at the position of the highest food concentration is recorded as well.

The adjustment of center with a help of swarming behavior of artificial fish:

The local optimum of training sample during the process of iteration is:


In the equation above,

The formula for adjusting the center:


Approaching the iteration n, we use sample as closest to current center to replace .

Denoting the weight with swarming behavior:


Where and represents the corresponding values of the component weights in the process of iteration from time to time .

According to the swarming behavior principles in the adjustment of the center we compare the results of center adjustment with previous iterations to determine error value . Thus, we assume the distribution of the center is approximately fixed and then update the optimum result in the report. Searching behavior in determining the weight value represents the comparing of the current food concentration with the one from report and update it in case the result is better. In order to make a decision whether to continue the iteration or stop it we apply to preset condition that food concentration is bigger than or the maximum iterative number is achieved. If conditions are satisfied results are sent to output. Otherwise, iterate again.

Finally we define , which is the center of primary radial basis function. The last step is to calculate the variance which is done with a help of Gauss function:


Where is the minimum distance between the chosen centers.

This hybrid K-means clustering procedure discussed above can be demonstrated with the following graph:

Fig 2. Hybrid K-means clustering procedure


In this work a new hybrid algorithm, radial basis function optimized by artificial fish swarming algorithm, is introduced. First, the artificial fish swarming algorithm is used to optimize a K-means clustering algorithm. Second, it is used to identify the linking weight of radial basis function. This hybrid approach is aimed to minimize the forecasting error of radial basis function and improve the accuracy. Besides, described clustering procedure is a useful and easy-to-apply method for parallel computation and is independent from initial value.


  1. [The Genius Within: Discovering the Intelligence of Every Living Thing by Frank T. Vertosick Jr.]

  2. [Swarm Intelligence by James Kennedy]

  3. [Swarm Intelligence: From Natural to Artificial Systems by Eric Bonabeau]

  4. [Complex-Valued Neural Networks: Utilizing High-Dimensional Parameters by Tohru Nitta]

  5. [Neural networks: a comprehensive foundation by Haykin, S. 1999]

  6. [A Set of Neural Network Benchmark Problems and Benchmarking Rules by Lutz Prechelt 1994]

  7. [Business Intelligence in Economic Forecasting: Technologies and Techniques by Jue Wang; Wang Shouyang 2010]

  8. [Handbook of Research on Computational Science and Engineering by J. Leng; Wes Sharrock 2011].

Основные термины: radial basis function, artificial fish, hidden layer, basis function network, k-means clustering procedure, hybrid k-means clustering, fish swarm algorithm, output layer, artificial fish swarm, Fish Swarming Algorithm, activation function center, training samples, input layer, radial basis functions, Machine learning, neural network, О. Hybrid K-Means, food concentration, basis function networks, problem artificial fish


Социальные комментарии Cackle