An information technique for segmentation of military assets in conditions of uncertainty of initial data

To solve the applied task of detecting military assets in aerospace images the presented paper investigates the processes of constructing segmented maps of the images. The goal is to develop an information technique for detecting military assets in conditions of uncertainty of initial data. To achieve the goal, the following tasks were formulated: 1) to analyze usability of the existing segmentation methods for automatic detection of military assets in the images; 2) if the existing methods are inapplicable, to develop a new algorithm to solve the problem. In the paper the following methods are used: the methods of digital image processing, the methods of Boolean algebra and fuzzy sets, the methods of statistical analysis. The following results are received. Analysis of the known segmentation methods showed that due to camouflage coloring of the military assets, similarity of their color characteristics to those of underlying surfaces and due to the presence of large number of textured fragments in the images those methods provide segmented maps of poor quality. Among the common problems arising when conventional methods are used there are wrong segmentation, when the received contours do not coincide with the borders of the objects of interest; oversegmentation, when there are a lot of minor segments which produce "litter" objects; undersegmentation, when potentially possible segments are missed etc. As the conventional methods are inapplicable, in the paper it is suggested using the fuzzy logic systems. For each pixel the probability of the fact that the pixel belongs to the object or to the background is calculated. For making decision whether a pixel belongs to the object the production rules based on the chosen most significant factors (probabilistic values of spectral sub-bands, belonging of the neighboring pixels to the object, jumps of brightness in spectral sub-bands on the object's borders) are constructed. Conclusion. The suggested technique ensures high-quality definition of objects' borders, thus considerably increasing the reliability of military assets recognition.


Introduction
Formulation of the problem.One of the most characteristic features of the modern warfare is the integrated nature of the surveillance process.Surveillance should promptly supply the political and military authorities and the armed forces with the necessary information with maximum completeness, accuracy and reliability.Demands for the data received from onboard systems of electro-optical surveillance constantly increase, which causes constant increase in general amount and availability of information with high spatial resolution and results in considerable time expenditures for processing of such information.Thus it is utmost relevant to automate the process of defining objects in images, as it should allow increasing promptness and efficiency of solution of intelligence, surveillance and reconnaissance tasks.Within the group of numerical methods used for processing of electro-optical surveillance data (Fig. 1) the following subgroups can be delineated: -The methods of improving transformations.In fact, these methods are an intermediate stage between visual and automated interpreting, as after their application the final result is not achieved.The aim of these methods is to improve the image and to prepare it either for visual or automatic interpretation.
-The methods of photogrammetric data processing.
Such methods imply finding interrelations between fragments in the image and their geographic coordinates.
-The methods aimed at constructing segmented maps.These methods consist in dividing the image into components with similar homogeneity criterion.Such components can represent both regions of objects and regions of backgrounds.
-Vectorization of objects on the basis of the built segmented map.It implies definition of boundaries between objects and a background or between objects and other informative fragments which have some descriptive signatures.
-Methods of automated interpretation.Classification of objects in the image is performed automatically either on the basis of a priory information about the signatures being selected or without it.For the time being there is no common theory for optimal representation and processing of images from onboard surveillance systems.Choice of some definite technique for image processing depends on the tasks being solved and the requirements posed to the processing results.Complexity of the image processing arises from [1][2][3][4][5]: -anomalies (small regions in the image) that can be taken for noise or some image defect; -unknown shape and blurred borders of the surveillance objects; -inhomogeneous structure of segments in the analyzed images.
The primary stage which defines the processing quality and the result of the automatic interpretation as a whole is the stage of thematic segmentation [4][5][6][7][8][9].Its main challenge is to achieve the 100% similarity between the initial and final images.Speaking of the thematic segmentation, we imply segmentation of an electro-optical image, definition of the surveillance objects signatures and semantic segmentation of the image.As the result we expect to divide the image into artificial (surveillance) objects and natural objects (background).
The segmentation quality is usually estimated by the following criteria: -homogeneity of regions (regions should be homogeneous in certain characteristics); -dissimilarity of neighboring regions (adjacent regions should be considerably different in the chosen characteristics in which they are homogeneous within the region); -smoothness of boundaries of the regions; -small number of little 'gaps' inside the region; -borders of each segment should be simple, spatially accurate.
There are a lot of image segmentation methods that use the basic properties of the brightness signal, its homogeneity and discontinuity.Such methods are oriented toward different division.
Analysis of the recent research and publications.In the process of segmentation edges of objects are defined and located.Edges are such lines in the image along which brightness or its spatial derivatives change abruptly.
According to the classification suggested by W. Skarbek and A. Koschan all segmentation methods can be divided into the following four groups [5; 10] -general approach; -inhomogeneous dielectrics.Each of the mentioned methods is oriented towards solution of some specific tasks.
Contour-based methods include various operators for contour analysis.Among the important simplest signatures defining contours of objects in the image there are abrupt changes in brightness, color coordinates or parameters that characterize texture.Local brightness shifts are called brightness jumps or brightness contours.Segmentation by contour detectors is performed globally, i.e. it does not define separate objects, though there is some possibility to introduce additional filtering configured on certain properties of the objects borders [11].An ideal jump detector should point to the presence of jump in a single point located in the middle of the slope.An initial image represented by an array of values ( , )  F j k is subjected to the linear or non-linear processing so that the brightness jumps should increase.
As the result the array of values is formed.The received array is the image with highlighted brightness changes.This image is compared with a threshold, and location of the image elements with explicit brightness jumps is defined.
then there is a descending jump, and when , ( , ) ( , ) the jump is ascending.The and values are the lower and the upper thresholds.These values can be made variable within the image plane to compensate the influence of considerable brightness changes.
( , ) Selection of the threshold is one of the key challenges in defining jumps.If the threshold is too high, the structural elements with low contrast won't be revealed.On the contrary, if the threshold is too low, noise could be mistaken for a jump.Influence of the threshold level to the results of contours detection is shown in Fig. 2; here the Sobel filter was applied.
Thus, when brightness jumps are revealed with the help of detectors, considerable drawbacks arise.Among them there are the dependence of the processing results on the quality of the image and thus the necessity of preliminary selection of filter parameters, the dependence of the results on the skills of the operatorinterpreter.Besides, the methods of contour analysis do not account information about spatial location of pixels.Linking of separate contour fragments is the task of further steps of processing, and the more highly textured objects there are in the image, the more complicated is the processing.In papers [12][13] for building segmented maps and defining contours of objects the genetic algorithms are suggested.
With genetic algorithms the solution of the problem is searched on the subset of the search space points, which is achieved by creation of the set of potential solutions.This set forms population.The population is improved by genetic operators which account for variability and fitness functions which imitate natural selection.Inheritance is ensured due to the fact that new chromosomes are formed from the chromosomes of the previous generation and consequently they have common genes.
If the genetic algorithm is realized correctly, with each new generation the average and the best values of the fitness function of the population are growing to the side of the global optimum.
Application of the adaptive genetic algorithm for solving the task of color image segmentation complicated by the necessity of making decision as to the optimal number of segments and the necessity of accurate definition of textured regions is discussed in [13].
Usage of the genetic methods for segmenting the electro-optical images of the Earth surface is not reasonable due to the following drawbacks [14][15][16]: -the genetic methods are highly iterative; -the quality of the results received by the genetic methods depends considerably on their parameters (population size, initial start-point, probability characteristics of genetic operators etc.); -presence of epistasis, which is an internal interdependency between genes coded in chromosomes; -premature convergence of the methods caused by insufficient variability of chromosomes in the population, which results in finding local optimums instead of the global one.
Papers [13; 15-17] suggest using the ant colony optimization that allows searching for the global optimum bypassing local ones and searching for several optimums (or suboptimal solutions) located in different points of the search space.
The general advantages of the ant colony optimization are: -disinclination to move in loops in local optimums; -multi-agent realization; -ability to get adopted to the changes of the environment; -can be used both for discrete and continuous optimization tasks; -the best solution is searched with account of the solutions of all the agents.Fig. 3 shows the results of application of the ant colony optimization for solving the segmentation task (the figures are taken from [13, pp. 123, 128]).To investigate the possibility of application of the ant colony optimization for solving the task of the camouflaged military assets detection, among others we processed the image shown in Fig. 4, a which is an image with a lot of highly-textured objects and an asset with camouflage coloring.
Analysis of the received segmented map (Fig. 4, b) shows that the suggested ant colony optimization in such cases is unreasonable.The discussed above methods of contour analysis are not inherently the segmentation algorithms, as they do not divide the image into segments on their own, but just form a gradient image received from the initial one.Thus the following step to the segmentation is defining contour points in the gradient image.Usually it is done by searching local maximums of the gradient or using the method of non-maximum suppression with the integrated threshold (from below) for the allowable gradient size.When contour points are defined, the challenge of their bounding into the continuous borders of segments arises, and the more textured objects there are in the images the more complicated solution of the task is.

Information Processing Systems, 2021, Issue 4 (167) ISSN 1681-7710
Segment-oriented segmentation methods at first define regions (segments) within which the values of the image function are homogeneous as compared to the set of background points or to the set of object points.At it the initial function values are compared with some threshold which in general case is defined for each pixel of the image: T 1, ( , ) , ( , ) 0, ( , ) ; where ( , )  f x y is the local characteristic of the ( , ) x y point.Usually brightness of the spectral sub-bands of the pixel color is used.
It is convenient to use the threshold transformation when it is possible to divide the background and the objects in the image, and the histogram has an explicitly bimodal character (Fig. 5).But general universality of such approach would be very poor should it be used for processing images with other characteristics of brightness and noise distribution.

Fig. 5. An example of a bimodal histogram
Source: developed by authors.When the segmentation methods based on analysis of the color characteristics are used, the main trouble is the segmentation of the highly textured objects [18][19].For the images of the Earth surface it is the case of distinguishing between forests (single plants) and steppes, meadows etc., as they have similar color characteristics but differ in texture.
Segment-oriented methods of region-based segmentation directly take into account spatial location of pixels.This group of methods includes: -the methods based on graph theory; -the watershed segmentation; -the split-and-merge method; -the region-growing method.
A general feature of these methods (except the method of split-and-merge) is that to define differences in segments and within the selected signature space it is possible to use the heterogeneity function The methods based on the graph theory [20][21][22] in general can be described as follows: at first the raster image is represented as a weighted graph, where one pixel corresponds to each vertex.Weights of the relations between the vertices represent the difference of the corresponding pixels in some signature (brightness, texture, spatial distance) or their combinations.To calculate the weights of the relations the heterogeneity function is introduced, its parameters are adopted for each application.In such model an optimal segmentation is equal to the minimum cut of the graph.In the original the methods of the graph theory were developed to distinguish objects from backgrounds, but not to perform full segmentation of the whole image, thus they require user participation in the segmentation process.The user should mark in the image some characteristic points that should belong to either the background or the object being detected.Besides, the user may need to introduce other segmentation parameters which influence considerably the result.When the graph methods are improved to be used for full segmentation in automatic mode, the challenge of definition of the optimal set (number) of segments arises, which approximates the methods of this group to the region growing methods.
The idea of the segmentation methods based on the morphological watersheds consists in representation of the image as a three-dimensional relief, where peaks correspond to the brightest points of the image (usually they are region contours), and valleys (basins) which are the regions of low intensity [23].When the algorithm is executed, it looks like water rises up the relief from the lower points (which are the points of local minimum).As far as the water level comes up, the flooded regions grow.At the moment when neighboring basins start merging, a watershed is put between them with the help of the mathematical morphology methods.When the algorithm finishes its operation, the whole image appears to be divided into segments.Then, if necessary, adjacent segments are merged.At it the points of the segments are marked anew and merged, and the characteristics of the new segments (average brightness of points, area, perimeter etc.) are calculated.To do that the graph theory is used as it is described above.
The method of regions split-and-merge is based on the pyramidal representation of an image [24].This method has rather simple description with possibility of compact formalization via the recursion.It fulfills full segmentation, uses information about the relations between pixels, but requires strict defining of predicate that should lead the algorithm to the correct segmentation for the given class of images.The method has low universality, as for the statistical characteristics of the segments different thresholds are introduced.That is why the method is commonly used in highly specialized technical systems and inapplicable for segmentation of the camouflaged military assets.

Системи обробки інформації, 2021, випуск 4 (167) ISSN 1681-7710
The region-growing methods are based on the following idea.At first, according to some rule, the seeds of regions are chosen (there exist different strategies for seeds selection and regions growing).Step-by-step neighboring points which meet some defined criterion join the seeds [25].The process of the region growing stops, when neither point of the image can be added to any region.To decide whether a pixel can be added to a region or cannot, various criteria are used, e.g.proximity (in some sense) of the point to the region seed; proximity to the neighboring point added to the region in the previous step; proximity according to some region statistics; cost of the shortest way from the point to the region seed etc.The region-growing procedure is used mainly to get individual regions, but using the procedure successively or simultaneously for several regions one can achieve segmentation of the whole image.The procedure for the region growing does not suffice the requirement to the smoothness of the region border (Fig. 6-7).Fig. 6.A vector image of the object received by the method of region growing according to the criterion of proximity to the neighboring point Source: developed by authors.Fig. 7.A vector image of the object received by the method of region growing according to the criterion of proximity to the region seed Source: developed by authors.
So, when the region growing procedure is used, the difficulties arise at vectorization of the textured objects and the objects with camouflage coloring.
On the basis of the carried out research we can conclude that one of the reasons that it is impossible to directly apply the known methods of thematic segmentation for detecting military assets in electro-optical images is that the known methods insufficiently take into account the peculiarities of building images by onboard surveillance systems.Besides, the mentioned methods have the following drawbacks: -Wrong segmentation, when the received contours do not coincide with the borders of the surveillance objects in the image -Oversegmentation, when there are a lot of minor segments which produce "litter" objects -Undersegmentation, when potentially possible segments of surveillance objects are missed -The majority of the methods do not define objects borders and do not perform segmentation, but just highlight objects borders; -Segmentation is performed just over the criterion of the pixel brightness, which does not allow defining the homogeneity criteria of the segments.
The goal of the presented paper is to develop an information technique for detecting military assets in the conditions of uncertainty of initial data.

Statement of basic materials
Analysis of the known segmentation methods showed that due to camouflage coloring of the military assets, similarity of their color characteristics to underlying surfaces and due to the presence of large number of textured fragments in the images those methods provide segmented maps of poor quality which does not satisfy the mentioned above criteria to the quality of the built segmented map.
As an example Fig. 9 presents brightness histogram for the spectral sub-bands of the object (1) and the background (2) for the image shown in Fig. 8.The result probabilities received according to (1) for the object shown in Fig. 8 are presented in Fig. 10.Fig. 10.Probabilities that define whether a pixel belongs to the object (1) or to the background (2) for the R, G and B color channels Source: developed by authors.
The probability maps built in different spectral sub-bands are shown in Fig. 11.As it can be seen out of Fig. 11, none of the sub-bands can provide unambiguous answer whether a pixel belongs to the background or to the object.The segmented map built on the basis of combination of the probabilities received for different spectral sub-bands does not satisfy the formulated above criteria of segmentation quality (Fig. 12).Thus to solve the task of military assets detection it is suggested using the fuzzy logic systems.To make a decision whether a pixel belongs to the object in the conditions of uncertainty about the color characteristics of the object it is relevant to formulate production rules and models.Usage of this method implies choice of the most significant factors for approving the decision whether a pixel belongs to the object image.
The most important stage for the system construction is the development and refinement of the criteria that should define whether a pixel belongs to the object image.It is rather difficult to develop a model that should most comprehensively represent such situation.The reason is that the conditions for joining pixel to the object or to the background are highly various.Factors influencing the decision-making are given in Fig. 13.Fig. 13.The influence graph for making decision whether a pixel belongs to an object image Source: developed by authors.The structure of the technique that can be used to build segmented maps and which defines the rules for making decision whether a pixel belongs to an object as well as the algorithm of the technique realization are shown in Fig. 14.The necessity to apply the set-theoretic models of the decision-making to each pixel of the image on the basis of Boolean algebra and fuzzy sets operation to receive interval estimates in conditions of uncertainty is caused by the ambiguity of color characteristics of the object and the background.Such models are constructed similarly; the difference is defined just by the set of initial data.According to the algorithm for the production rules construction (Fig. 14), out of the formal representation of the empirical knowledge depending upon the chosen factors, the following 27 fuzzy logic rules were received: where R, G, B are the probabilities of pixel F(x,y) belonging to the object in the R, G and B channels; Р1, P2, Р3 are the 'integral' probabilities (account the information for all the three channels) that the pixel belongs to the object: Р1 -high probability, P2 -average probability, Р3 -low probability.
Correspondence between the numerical values of the input variables of the fuzzy logic system and the values of the membership function is revealed be means of fuzzification.For the term membership function the triangular membership functions were used.Each function is given within the universum X=[0, 1].Fig. 15 shows the chart that defines the membership function ( ) P x  for all the term-sets depending upon the pixel color.16).Having defined the points in which the functions intersect (Fig. 17) let us describe analytically the uncertainty areas of the functions depending upon the chosen factors.To do that we use the equation of the line:   To automate the process of the membership function calculation the expressions that analytically de-scribe the characteristic curves for the uncertainty areas of the qualitative terms membership functions depending upon the color with the intersection points coordinates [0.The following stage of the fuzzy inference according to the algorithm is the stage of aggregation.The aggregation operation allows defining the conditions truth degree for each rule of the fuzzy logic system.As all the rules for the sub-conditions use fuzzy disjunction, it is relevant to use the operations of maxdisjunction for the method of aggregation: ( , ) max ( 1 2 3) F x y P P P    .
The rules with the highest confidence degree for the corresponding conditions are considered active and are used for further calculations.
The most reasonable method of defuzzification which can be used to receive the numerical value of the integrated estimate that characterizes the probable belonging of the ( , )  x y pixel to the object is the center of gravity for singletons method: where is the result of the defuzzification; Dz

Conclusion
The logic function changes an L 0 A pixel value from 1 to 0, if in its neighborhood there is no single pixel with the value 1, and from 0 to 1, if all the neighboring pixels are equal to 1.The function takes the form: 8 ,

L A A A A A A A A A A A A A A A
The first and necessary stage that defines the processing quality and the result of automatic interpretation as a whole is the stage of thematic segmentation of the image.Its main goal is to achieve 100% similarity between the initial and final images.The quality of segmentation is usually estimated according to the following criteria: homogeneity of regions; dissimilarity of neighboring regions; region boundaries should be smooth; there should not be many small 'gaps' inside the region; boundaries of each segment should be simple, spatially accurate.Analysis of the known segmentation methods showed that due to camouflage coloring of the military assets, similarity of their color characteristics to those of underlying surfaces and due to the presence of large number of textured fragments in the images those methods provide segmented maps of poor quality (see, for example, Fig. 4, Fig. 6, Fig. 7).
where is the logical negation (NOT);   is the conjunction (logical AND);  is the disjunction (logical OR).When larger neighborhood is used, the logical function (2) becomes more complicated, and the time necessary for image processing increases.
At the end of the segmentation process to increase the accuracy of the object borders definition it is relevant to use the contour-highlighting Sobel filter which ensures high degree of distinguishing between the noise and the structure objects of the image.
The result of the image processing according to the suggested technique is shown in Fig. 19.
As the conventional methods are inapplicable, in the paper it is suggested using the fuzzy logic systems.For each pixel the probability of the fact that the pixel belongs to the object or to the background is calculated.For making decision whether a pixel belongs to the object the production rules based on the chosen most significant factors (probabilistic values of spectral subbands, belonging of the neighboring pixels to the object, jumps of brightness in spectral sub-bands on the object's borders) are constructed.
To increase the segmentation quality the method based on definition of the function that is calculated in the neighborhood of the zone classified as a certain class is used.The logical function (2) depends upon the spatial location and values of the neighboring regions, and can take the values 0 or 1. a At the end of the segmentation process to further increase the accuracy of the borders definition it appeared to be reasonable to use the contour-highlighting Sobel filter which ensures high degree of distinguishing between the noise and the structure objects of the image.b Analysis of the received results (see Fig. 19) lets us conclude that the suggested technique ensures highquality definition of objects' borders, thus considerably increasing the reliability of military assets recognition.

Fig. 1 .
Fig. 1.Generalized diagram of the stages used to process electro-optical images Source: developed by authors.

2 .
The results of application of non-linear contourhighlighting Sobel filter with various threshold values: а -initial image; b -the threshold equals 100; c -the threshold equals 150 Source: developed by authors.

Fig. 3 .
Fig 3, a is the initial image, Fig 3, b is the segmented map.а b The results of thematic segmentation on the basis of the ant colony optimization: а -initial image; b -result image Source: [13, p. 123, 128].

4 .
The results of thematic segmentation on the basis of the ant colony optimization of high-textured objects and military assets: а -initial image; b -result image Source: developed by authors.

Fig. 11 .Fig. 12 .
Fig. 11.The map of probabilities that a pixel belongs to the object built in different spectral sub-bands Source: developed by authors.

Fig. 14 .
Fig. 14.The structure of the technique that can be used to build segmented maps Source: developed by authors.

Fig. 15 .
Fig. 15.The dependence of the qualitative terms membership functions upon the pixel color Source: developed by authors.Let us define the uncertainty areas in the charts of the membership functions depending upon the brightness of the spectral sub-bands (Fig.16).Having defined the points in which the functions intersect (Fig.17) let us describe analytically the uncertainty areas of the functions depending upon the chosen factors.To do that we use the equation of the line:

Fig. 16 .
Fig. 16.The curves representing the uncertanty areas for each spectral sub-band Source: developed by authors.

Fig. 17
Fig. 17.The membership functions with marked points of their intersection for each of the factors Source: developed by authors.
Fig. 17.The membership functions with marked points of their intersection for each of the factors Source: developed by authors.

iX 1 A 2 A 3 A 4 A 0 A 5 A 6 A 7 A 8 AFig. 18 .
Fig. 18.Symbolic representation of the neighborhood of the region 0 A Source: developed by authors.

Fig. 19 .
Fig. 19.The result of the image processing according to the suggested technique: а -the segmented map; b -the revealed military assets Source: developed by authors.