EFFICIENT LINE DETECTION METHOD BASED ON 2D CONVOLUTION FILTER

The article proposes an efficient line detection method using a 2D convolution filter. The proposed method was compared with the Hough transform, the most popular method of straight lines detection. The developed method is suitable for local detection of straight lines with a slope from -45 ̊ to 45 ̊. Also, it can be used for curve detection which shape is approximated with the short straight sections. The new method is characterized by a constant computational cost regardless of the number of set pixels. The convolution is performed using the logical conjunction and sum operations. Moreover, design of the developed filter and the method of filtration allows for parallelization. Due to constant computation cost, the new method is suitable for implementation in the hardware structure of real-time image processing systems.


Introduction
Colinear pixels detection is often used in image processing. For example, for recognizing objects consisting of straight edges. The most commonly used method is the Hough transform [8]. It is characterized by high efficiency and low sensitivity to disturbances, as well as insensitivity to line discontinuity [5,6]. The main part of this method is the voting procedure, in which each input image pixel is transformed into Hough space. Such a transformation requires determining a set of lines passing through a given pixel. It allows performing some kind of voting in two-dimensional Hough space. The voting results contain information about the number of pixels lying on each straight line. The high efficiency of this method and low noise sensitivity resulted in its modifications for the detection of more complex shapes. For instance, curve or circles detection [11,12]. This is commonly achieved by extending the Hough space and the voting table to three dimensions. Similarly, the method can be extended to other shapes by broadening the number of dimensions, but this escalates computational cost [1,8,11,12].
Hough transform can also be done by neutral networks, where it also extends the possibilities of detection [4,7,10] but due to multiplications during processing it may escalate computational cost.
In another approach, the method itself is not modified but is used for local detections. This approach allows the curve detection if the curve has got a slight curvature and locally is similar to a straight line. The computational cost of the global and local Hough method is similar. Detecting curves requires additional steps to connect the detected straight pieces together. This approach is often less computationally expensive than using the three-dimensional Hough method.
In both cases, broadening the dimensions of the voting matrix or connecting local parts together, allows the detection of more complex shapes, but enlarges the computational cost. The aim of the proposed method is reducing the computational cost compared to the standard Hough transform. The proposed method is dedicated to detect straight sections with an slope from -45˚ to 45˚. The main part of the proposed method is a 2D filtration with a developed set of masks. It is performed using the logical conjunction and sum operations.

Cartesian space
Two-dimensional Cartesian Space allows to describe the position of a point by two coefficients (x,y). The origin of the coordinate system is represented as a point (0,0). By Cartesian space coefficients, it is possible to determine the position of all pixels in an image. A line in Cartesian space image can be represented as set of pixels described by formula (1) = + .
(1) Also, the line can be determined by the perpendicular vector Ψ connecting the straight line and the point (0,0). Such a vector has got the constant beginning (0, 0), so Ψ can be described using two coefficients: α and ρ, where α is the angle of the vector Ψ, and ρ is the length of the vector Ψ (Fig. 1).

Fig. 1. Diagram of a line and Ψ vector in Cartesian space
The relation of the angle α and the coefficient a is defined in the formula (2) = − ( ) (2) The length of the vector Ψ is defined as ρ (3)

Hough space
The Hough space is two-dimensional space closely related to the Cartesian space. The line from Cartesian space is established by a single point in Hough space. Such a point is defined as a pair of coefficients α and ρ. This representation allows voting in the Hough space.

Voting in the Hough space
The main part of the Hough transformation is the voting procedure, where each pixel in the Cartesian space votes on the lines that pass through it. A single line is defined as a pair α and ρ and calculated by point position x, y and specified α. The set of lines is determined by specifying a set of α coefficients. It is common practice to determine the set of angles α divisible by the selected step value. For example, selecting step = 1˚, each pixel of the Cartesian space will vote on 360 different lines. Saving the votes is performed by increasing the value of the cell (α, ρ) of the Hough space matrix. The result of the vote is a twodimensional Hough space matrix, where the cell position dictates α and ρ and the cell value contains the number of votes.

Inverse transformation
The transformation of the vector (α, ρ) into the form (1) is called the inverse transformation. In this case, the coefficient a is defined by relation (2), and the coefficient b is determined by relation (4) = (4) Using the formula (1) allows determining all the pixels from the Cartesian space lying on the line. It can be used to draw this line in the image, for instance by changing the colour of these pixels.

New method
The new method was designed for the local detection of straight lines. For this purpose, the image is divided into vertical sections (Fig. 2). Straight lines are detected in each of these sections. Connecting short lines allows to identify straight lines and curves. In the new method, the multiplication of the real numbers was eliminated. There is also no need to determine the values of the trigonometric functions. In the presented method, calculations are performed in the form of a two-dimensional filter using a predefined set of masks. Each mask contains a fragment of one straight line. Convolution of the masks and the window is achieved using logical conjunction. The result is a set of bits. The number of set bits in the string indicates the number of pixels lying on the straight line, which is similar to the results in standard Hough method.

Masks
The masks are designed to detect lines with slope from -45˚ to 45˚. A single mask is expressed by a binary matrix of the size × ℎ , where ℎ = 2 − 1. Each mask has been marked with identifiers id. The masks are created using (5) where , is the cell (x,y) of the mask ℎ . Pixels lying on the line (5) take the value 1, and the rest of them the value 0. A graphical representation of the set of 15 masks of size 8 × 15 is shown in Fig. 3. The set bits are represented by white pixels.

Convolution
The convolution is performed by moving the window W around the images and convoluting with a set of masks. Image is divided into sections (Fig. 2), where single section can be treated as image S with a width of pixels. The size of the window W and the mask M in the presented algorithm are identical ( × ℎ ). While processing, window W is moving in section S only vertically, so its position can be defined as single variable j. Each pixel in window is described by formula (6) , = , + − +1 (6) where , is the single pixel of window , j is the window position, (x,y) is the position of the pixel in the window and , + − +1 is the pixel , + − + 1 of the section S. The convolution with single mask was divided into two steps.

First, the intermediate result is obtained by a logical conjunction of binary matrices M and W (7)
= ˄ (7) Then the set bits in C are counted. It should be noted that the mask M includes only set bits, so the result of the conjunction (7) can be saved in a sequence of bits. The next step is counting set bits in the received binary string. Methods of counting bits was presented in section 2.3. The results from all convolutions are stored in table R in form presented in formula (8) where id means the mask id, j the position of the window W in section S, and cb the function of counting set bits in a binary string.

Counting set bits
The most time-consuming part of the new method is counting set bits in the binary string. In the simplest approach, the bits are added sequentially. For the n bit sequence it requires a n-1 summations.
Another example of a sequential algorithm is the Brian Kernighan algorithm [9]. Execution of this algorithm requires 4•z elementary operations to determine the number of set bits, where z is the number of set bits in the string.
The quickest way is to store all results of counting bits of n bits strings in an array called a lookup table. Then the determination of the number of set bits is performed by reading the value directly from the memory. But it may require large amount of memory to keep the lookup table.
Counting the set bits may also be performed using a parallel algorithm. In this approach, all pairs of cells will be summed in each cycle. In each cycle, the number of cells summed is doubled. In that case it requires ⌈ 2 ⌉ cycles.
Another approach is a hybrid method combining techniques with the use of partial results, called lookup table and the simple sequential algorithm. In the hybrid method, the input string is divided into sections of m bits. For each section, the number of set bits is determined by simply reading from the lookup table. It takes ⌈ / ⌊ 2 ⌋⌉ operations to obtain ⌈ / ⌊ 2 ⌋⌉ subsets. Simple addition of these results needs ⌈ / ⌊ 2 ⌋⌉ − 1 operations. All in all, this procedure IAPGOŚ 4/2021 p-ISSN 2083-0157, e-ISSN 2391-6761 for counting the set bits takes 2 ⋅ ⌈ / ⌊ 2 ⌋⌉ − 1 elementary operations. It is faster than simple summation for ≥ 3. Another way to count bits is a combination of a parallel technique and a lookup table. In this case, the string is divided into m bit packets and the partial results are determined by parallel readings from the lookup table. The counting of bits in each section is also performed parallel. In this case, it takes ⌈ 2 ⌈ / ⌉⌉ + 1 cycles and requires ⌈ / ⌉ threads.

Searching for straight lines in the results
The detection of straight lines for both, the standard Hough method and the new method is similar. In both cases, the cell in the result array represents a single straight line, and the value determines the number of pixels lying on that line. Searching for straight lines is usually performed by thresholding the result table [6]. There is slight difference is the size of the resulting arrays of standard Hough method and new method. In the case of Hough, it is ℎ × √ ℎ 2 + ℎ 2 cells, and in new method, it is ℎ × ℎ .

Computational cost comparison of the new method and the standard Hough transform
The new method is dedicated for a narrow range of applications. Therefore, both methods are compared in detecting lines with ℎ different slopes from -45˚ to 45˚. In the new method, the image S is divided vertically into sections of pixels width (Fig. 2). Each section can be treated as a separate binary image of size ℎ × . A similar computation will be performed for each section, so the single section processing cost will be compared. The comparison was divided into three sections, where memory cost, calculation cost and parallel efficiency were compared.

Memory cost
In both algorithms, a significant amount of memory is occupied by the result table and the lookup table. In the case of the standard Hough transform, using a lookup table to store all necessary sin and cos values reduces the computational cost [2,3]. The size of such an array is 2 ℎ . Additionally, for each section, one voting matrix of the size of ℎ × √ ℎ 2 + ℎ 2 cells is created. The total memory cost is equal to 2 ℎ + ℎ √ ℎ 2 + ℎ 2 cells.
In the case of the new method, for the comparison the lookup table for counting set bits in hybrid counting algorithm will be same size as the lookup table of the Hough method, 2 ℎ .
The processing results of each section are kept in the matrix of size ℎ • ℎ . The total cost is equal to 2 ℎ + ℎ • ℎ memory cells. It can be seen that the new approach has got slightly lower memory requirements. The results of the memory cost comparison were shown in Table 1.

Calculation cost
The exact calculation cost depends on the platform, which is being used to run the method. For comparative purposes, the impact of platform constraints on the results, such as the time of reading data from memory, was ignored. The computational cost was determined as the number of elementary operations, where each operation such as addition, multiplication or comparison costs one clock cycle.
The standard Hough transform requires three procedures. First, the set bits are searched for. This requires the • ℎ comparison operations. For each of the z bits found, a set of ℎ straight lines is determined. This means computing a set of ρ coefficients. Each demands two multiplications by the sin and cos values from the lookup table and single sum operation (3). It means that determining the set of coefficients ρ takes ℎ • • 3 elementary operations. Determination of coefficients α is not included in the cost, because they are constant. Each pair of coefficients α and ρ votes by incrementing one cell of the result matrix. This requires the • ℎ increment operations. In total, performing the Hough transformation requires • ℎ + 4 ℎ • basic operations.
The new method is performed by the convolution of window size × ℎ with the set of ℎ masks. After each convolution the window moves one pixel down. Processing the entire image requires ℎ • ℎ convolutions with masks. Convolution of the window with the single mask requires a single conjunction operation and then the set bits counting. The hybrid algorithm with lookup table of size 2 ℎ performs counting in 2 ⋅ ⌈ / ⌊ 2 (2 ℎ )⌋⌉ − 1 elementary operations. So processing the image ℎ × with the new algorithm requires ℎ • ℎ • (2 ⋅ ⌈ /⌊ 2 (2 ℎ )⌋⌉ − 1) basic operations. Fig. 4 shows the dependence of the number of the set pixels on the computational cost of the processing a single image section of size 32 × 1080 pixels. In the standard approach, the number of operations is proportional to the number of set pixels z. In contrast, the number of operations for the new method is regardless of the number of set pixels. For the amount of about 6% set pixels, the number of operations of both methods is equal. Above this threshold (6%), the new method requires less operations than the standard method. But below the threshold, the new method requires more operations than the standard method.  It should be noted that the standard method involves the multiplication of the real numbers, and the new algorithm uses only conjunction operations and the addition of integers. The standard approach requires a procedure of set pixels searching, determining ρ, and voting. Parallel search for set pixels in one cycle requires × ℎ threads. Each thread performs one comparison operation. Once the z set pixels are found, for each set pixel ℎ coefficients ρ are determined. It requires • ℎ threads and 3 clock cycles. Then, each of the threads votes. Multiple threads will modify one cell of the result array, this requires thread synchronization. Even using memory, which allows parallel access to each memory location, the blocking time is equal to ℎ operations.
In the case of the new method, the image processing is performed by the convolution procedure. It is composed of the conjunction of windows with masks and counting of set bits. This requires ℎ • ℎ threads. Each thread will perform a single conjunction operation. Then, the set bits are counted. Parallel counting requires ℎ × ℎ • ℎ /2 threads and 2 ℎ clock cycles. An alternative is the hybrid method, which requires ℎ • ℎ threads and 2 ⋅ ⌈ / ⌊ 2 ( ℎ )⌋⌉ − 1 clock cycles. Each result is stored in a different memory location, so all results can be saved simultaneously. It requires ℎ • ℎ threads and a single clock cycle. The results of the comparison were depicted in table 2. Hough, > ℎ /2 With the parallel Hough transform and fully parallel new method, the number of threads varies during processing. Moreover during the voting procedure of parallel Hough transform the threads sleep most of the time. By the contrast, with the new hybrid algorithm, a fixed number of threads is used throughout the entire process, and no one thread is sleeping at any time. Such effective use of computational capabilities makes this method suitable for hardware implementation in a real-time image processing system.

Line extension in the next section
The input image is split into sections with the width of (Fig. 2). Lines are detected independently in each section. It makes the detection of the curved lines possible. The shape of a curved line can be approximated as many short straight lines, each in a separate section (Fig. 6).
To obtain the curve, the appropriate short sections must be connected. The convenient approach is to divide the image with sections overlapping by 1 pixel. In this case, the absolute position of the last pixel of the line in section ( ) is the same as the first pixel of the connected line in the section ( +1) . In order to find a connected line, first the position of the rightmost pixel of a given line should be calculated. Then, the straight line from the next section going through that pixel should be found. The cost of this procedure was compared for the results from standard Hough method and the new method. In the case of the Hough method, each section is associated with the Hough space voting results. The rightmost pixel of a line (α,ρ) in a section ( ) can be found by inverse transformation and using the formula (1). After locating this pixel in the next section ( +1) , it needs to be transformed to the Hough space getting ℎ lines passing through this pixel. Then it should be checked, which of these lines has the most votes in the result table of section ( +1) . Such a procedure requires two operations for the inverse transformation, two operations for determining the point position in the section ( +1) , 3 • ℎ operations for the Hough transformation, and ℎ operations for checking the voting table.
In the case of the new algorithm, the results from the section ( ) are stored in the array R of size ℎ × ℎ . The straight line , means that the line segment starts at the point (0,y) of the section and ends at the point ( , − + ). The continuation of the , line from the ( ) section is − + , from the ( +1) section, where ∈ < 0; ℎ − 1 > indicates ℎ lines with different slopes. This procedure requires two operations to find the pixel in the next section and ℎ − 1 operations to find the straight line with the highest score. In total, it requires ℎ + 1 basic operations.
Straight line extension search for the standard approach requires an 3 • ℎ + 4 operations. However, in the case of the new algorithm, it requires only ℎ + 1 operations. In this case, the new method is much faster.

Check if a line from the result table passes through the selected point
In the case of the Hough method, the line is described by coefficients , and in case of new method by , . The basic operations is to check if a given point = ( , ) from the input image is part of the selected line.
In the case of the standard Hough transform, it consists in the inverse transformation and checking if the pixel P matches the obtained formula (1). This requires two operations for the inverse transformation and three operations to check if the point matches the formula. In total, this requires 5 basic operations.
In case of new method, checking if a line from the result table passes through the selected point requires inserting a pixel P into a blank window. It can be performed by single assignment operation. Then, the conjunction of this window with the mask is made and check if the obtained result contains set bit. This procedure requires one operation to insert a pixel into the window, a single operation of window and mask conjunction, and a single operation to check if the result contains set bit. In total, this requires 3 basic operations. In both cases, the check can be done quickly.

Drawing the line in Cartesian space
After lines detection, drawing found lines in Cartesian space image is the common procedure. In the case of the standard Hough method, the inverse transformation should be performed. Then, for each x factor of the image, the y factor is calculated using the formula (1), and the pixels are changed to make a line visible. The inverse transformation requires 2 operations. Determining the set of points (x,y) requires 2 • operations. In total, this requires 2 • + 2 basic operations. In the case of the new algorithm, masks can be used for drawing. A single line in the voting matrix corresponds to a single mask. Drawing this line can be executed by logical disjunction of the appropriate mask and the appropriate part of the image. It can be done by a single logical operation. It is much faster than drawing a line based on standard Hough transformation results.

Parallel lines search
One of the common procedures of the detected lines processing is searching for parallel lines. In the case of standard Hough transform, lines 1 = ( 1 1 ) and 2 = ( 2 2 ) are parallel if 1 = 2 . Finding lines in Hough space with identical α coefficient can be done by a single column search of the result matrix. It requires √ ℎ 2 + ℎ 2 comparison operations.
In the case of the new algorithm, The lines 1 = ( 1 , 1 ) and 2 = ( 2 , 2 ) are parallel if 1 = 2 . Finding the lines with identical id can be done by checking a single column of the result matrix. This requires ℎ comparison operations.
In both cases, the search is performed with only comparison operations. The standard method requires √ ℎ 2 + ℎ 2 comparisons, and the new method ℎ comparisons.

Experiments
The developed method was used to detect overhead lines in photos with an environmental background. The test procedure is composed of the Canny edge detection algorithm and the developed method, which divides the image into sections 64 pixels wide. Straight lines are detected in each section. The results of processing sample photos are shown in Fig. 7, 8 and 9. Each example contains three images: a raw image (a), an image after edge detection with the Canny method (b), and an image with straight lines detected by the developed method (c).

Conclusion
The paper presents a new method of collinear pixels detection. The method is suitable for detecting straight lines with a slope from -45˚ to 45˚. An example of the method's application is detecting curves whose shape is locally approximated using short straight lines. The proposed method is characterized by a constant computational cost regardless of the number of set pixels. It is achieved by two-dimensional filtering using a dedicated set of masks performed using conjunction and sum operations. It eliminates real numbers multiplication. Due to constant computation cost, the new method is suitable for hardware implementation in a real-time processing system. The computational cost of selected operations performed after detecting straight lines was also compared. It has been shown that using the results of the new algorithm, it is faster to perform operations such as:  line extension in the next section,  checking whether a line passes through a selected point,  drawing a line in the Cartesian space,  searching for parallel lines. The comparison was presented in table 3.