To evaluate the network's success in becoming sensitive to phonotactic constraints, we first need to judge how well it predicts individual phonemes. For this purpose we seek a threshold above which phonemes are predicted to be admissible and below which they are predicted to be inadmissible. This is done empirically - we perform a binary search for an optimal threshold, i.e. the threshold θ* that minimizes the network error E(θ). The classification obtained in this fashion constitutes the network's predictions about phonotactics.
We now turn to evaluating the network's predictions: the method to evaluate the network from this point of view compares the context-dependent network predictions with the corresponding empirical distributions. For this purpose, the method described by Stoianov (2001) will be used. The algorithm traverses a trie (Aho, Hopcroft & Ullman, 1983: 163-169), which is a tree representing the vocabulary where initial segments are the first branches. Words are paths through this data structure. The algorithm computes the performance at the optimal threshold determined using the procedure described in the last paragraph, i.e., at the threshold which determines which phonemes are admissible and which inadmissible (see also 2.1). This approach compares the actual distribution with the learned distribution, and we normally use the complete database LM for training and testing.
Figure 2 shows the error of SRN18 0 at different values of the threshold. The optimal threshold searching procedure resulted in 6.0% erroneous phoneme prediction at a threshold of 0.0175. This means that if we want to predict phonemes with this SRN, they would be accepted as allowed successors if the activation of the correspondent neurons are higher than 0.0175.
Dostları ilə paylaş: |