Semi-automatic Segmentation & Alignment of Handwritten

Performance Experiments

To test the performance of the two developed algorithms and how well Bayesian opti- misation works to output reasonable parameters, the Labour’s Memory data set and the IAM data set were utilised to give a broader understanding of the performance.

The data were selected using a randomised method to avoid bias in the performance evaluation. 20 images were randomly picked using the random python package. The same 20 images were used across all experiments to compare the methods.

The images in IAM required additional preprocessing steps as the word bounding boxes were not complete, but rather split up into components. This was solved by taking the extreme coordinates of each word component to get a complete word box. Furthermore, the images contained a digital transcript above the handwritten text. The handwritten text was encapsulated by cropping the image to the top and bottom line y-coordinate and adding a margin.

The initial experiment aimed to assess the performance of the two algorithms under the condition of manual parameter configuration of {t1, min_gap}. For each image, the parameters were reasonably set through the interactive window and slider features. The GT is used for comparison to get a comprehensive understanding of the segmentation performance. A comparative analysis was performed between the GT count of lines and words and the count of lines and words obtained through the segmentation process for both developed methods. In this, the error could be calculated according to Equation 2. Note that all error metrics are calculated using the absolute value.

Seg_boxes

Error = |1 − _{GT_W ords} | (2)
IoU-based alignment was evaluated by calculating two metrics. The number of aligned words was counted, and the error could be calculated according to Equation 3.

W ords

Error = |1 − _{GT_W ords} | (3)
Each aligned word was compared to that specific word in the GT by calculating the IoU between the two boxes. The mean IoU was calculated for all aligned words in each image to better represent the overall performance. This ensured that boxes retrieved from segmentation faults were not part of the result, as they were filtered out.



×
To measure how well Bayesian optimisation worked to set parameters, the same 20 im- ages chosen for each data set were run through the Bayesian process for methods 1 and 2, respectively, for a total of 4 20 sets of parameters. These parameters were used to calculate the same metrics as in the manual experiment to compare them.

As the GT for Labour’s Memory is not as accurate as needed for a fair assessment of performance, one image was randomly chosen for the creation of a newly made GT to convey the faults of the original GT. The newly made GT for the image was made through the developed algorithm. Where correction of a box was needed, interactive correction was used (Chapter 4.4). Worth noting is that since the developed algorithm was used for making the new GT, it is prone to introduce a bias favouring the developed algorithm.

Below are the metrics calculated for each experiment, with a short explanation.

• 
  IoU: The final alignment IoU value. Each aligned word in an image is compared to that word in the GT; if a word is in the GT but not in the final alignment, this was skipped in the calculation. A word was skipped if there was no segmented box that this word overlaps with at all.
  

•