Studies in Big Data Supporting Web Search and Navigation by an Overlay Linking Structure

Search and navigation in hyperlinked networks have been subjects of research since the Internet emerged. Due to its incompleteness in terms of linking related content, the existing linking structure of the Web and similar networks cannot be utilized as a searchable index without prior application of suitable crawling strategies and content categorization. Following the example of sitemaps, a map-like extension to the existing link structure of the network is proposed that creates additional contextual links. For this, a concept and algorithms are devised that allow the creation of contextual cluster files, to which documents are assigned and between which semantically relevant links are established. The resulting WebMap covers all searchable resources on the original network in a contextual overlay network and enables new search and navigation approaches.