Information extraction from the web using a search engine Citation for published version (apa)
Yüklə 0,9 Mb. Pdf görüntüsü
|
|
O(s
0 · s 1 ). Since in worst case we need to compare each pair in P, we consider this as computationally too expensive. Creating a Fingerprint for the Lyrics Instead of comparing the full strings in the set P, we compare the fingerprint of the strings. We use the assumption that longer words occur less frequently in texts than shorter ones [Manning & Sch¨utze, 1999; Sigurd, Eeg-Olofsson, & Van Weijer, 2004]. We hence define the fingerprint f (t) = (w t1 , w t2 , . . . , w tm ) as the m longest – and most typical – words for a text t. For these purposes we define the order ≺ on words as follows. w 0 ≺ w 1 if and only if - The length of w 0 is smaller than the length of w 1 , or - w 0 and w 1 have equal length and w 0 is lexicographical smaller than w 1 . The last criterion assures the selection of fingerprints from the texts to be con- sistent. 98 The Beatles - Penny Lane photographs roundabout meanwhile hourglass suburban Bob Dylan - A Hard Rain’s Gonna Fall executioner’s whisperin mountains graveyard forgotten O-Zone - Dragostea Din Tei fericirea dragostea primeste amintesc Picasso The Police - Roxanne wouldn’t tonight streets Roxanne another Prince - Purple Rain underneath friendship something laughing changing Procol Harum - A Whiter Shade of Pale straightway cartwheels cardboard wandered straight Queen - Bohemian Rhapsody Thunderbolt Scaramouche monstrosity frightening silhouetto Britney Spears - Baby One More Time loneliness something shouldn’t wouldn’t supposed Emily Bront¨e - Wuthering Heights intercommunication mispronunciations incomprehensible unsatisfactorily unconsciousness Charles Dickens - Oliver Twist pockethandkerchief chimbleysweeper’s stauncherhearted unconstitutional Northamptonshire Euclid - The Elements (book 1) parallelogrammic quadrilaterals Parallelograms quadrilateral perpendicular E.W. Dijkstra - Goto statement considered harmful superfluousness undesirability specifications recommendation correspondence Table 5.15. The fingerprints, with f = 5, of a number of texts. As a fingerprint for t we thus select the m longest words using ordering ≺. The fingerprint of t can be computed in time linear in the length of t. Table 5.15 gives examples of the fingerprints for a number of lyrics and other texts. Comparing Lyrics using Fingerprints Having computed the fingerprints f (t) for each text fragment t, we use them to select a subset P 0 of P such that each t in P 0 is a representation of the lyrics of the song queried. As an example, Figure 5.1 gives the fingerprints of 12 texts gathered for the song Silver and Gold by U2. The fingerprint of text 8 contains indeed terms that occur in the lyrics of the song. However, as the fingerprint words are shorter than the ones of e.g. text 1, text 8 will be an incomplete version of the lyrics to this song. Texts 1 and 6 share three out of five fingerprint words, where text 6 will not contain the word silence. The fingerprint for text 11 reflects a different song as it 5.2 Extracting Lyrics from the Web 99 1 temperature something prisoners daylight silence 2 temperature something prisoners daylight silence 3 temperature something prisoners daylight silence 4 satisfied important seriously caution foolish 5 temperature something prisoners daylight silence 6 temperatur something prisoners daylight someone 7 temperature something prisoners daylight silence 8 shotgun praying hunter hunted house 9 rivers lights silver these shine 10 temperature something prisoners daylight silence 11 punctures waterless disappear carnival tonight 12 temperature something prisoners daylight silence Figure 5.1. The fingerprints of 12 lyrics versions of the song Silver and Gold by Yüklə 0,9 Mb. Dostları ilə paylaş:
|