Semi-automatic Segmentation & Alignment of Handwritten

Self learning

An attempt to make the algorithm more automatic was developed with the use of Bayesian optimisation. The goal of using this method was to find and automatically set optimal values for the parameters {t1, min_gap}, which usually requires manual input. Bayesian optimisation requires a black box function with a numerical output to be maximised and parameters in the black box function to be optimised for. The func- tion is, in this case, the segmentation algorithm as described above but with a modified output. The output is treated as a heuristic value and indicates to the optimiser how good the segmentation is. The heuristic is based on comparing the value of the IoU of a newly segmented box to all individual GT boxes and taking the maximum value found. By calculating the maximum value for all individual segmented boxes and taking the mean of all these, a value in the range of [0, 1] is acquired. An output value of 1 corresponds to a perfect match to the GT, whilst an output value of 0 indicates a complete absence of overlapping boxes between the segmentation and the GT.

The settings for the Bayesian process are described as follows:

For the utility function, kappa=1.5, for the internal Gaussian Process Regressor,

alpha=1e-8, Number_of_iterations=20 including a starting probe with parameters
{t1=205, min_gap=33}.
The maximum output heuristic found in the 20 iterations and the parameter values
{t1, min_gap} for this iteration are saved in an XML file. This XML file can be chosen to be read in the alignment algorithm to not have to manually input parameters.

This Bayesian process can be streamlined by submitting batch jobs for the images that are to be processed.

6. User interface

A concept application with a suitable graphical user interface was developed to show how the algorithm could be used in an online platform environment. Anvil (Luff & Davies 2023) was chosen as the framework as it is an open-source, free-to-use, and Pythonic platform. The concept was to have an easy-to-access collection of images stored and be able to align them to a GT; with an emphasis on the visualisation of the process by showing how the image is transformed in each step while being user-friendly. Images to be processed can be uploaded to the online gallery collection (Figure 14).

1. 
   Gallery
   

2. 
   Segmentation tool
   

Figure 14: The pages of the Anvil concept application.

5.