The Main Search Engine Functions

For more than 15 years now, internet marketing has become a strong force to reckon on. Various forms of online marketing tools are used by companies from diverse numbers of industries. One can easily mention email marketing, affiliate marketing, article marketing and most recently social media marketing. Search engine marketing leads in terms of the number of users that utilize it as a source of finding quality products, services and information. It is the primary avenue for driving traffic to a website all over the world. There are so many search engine platforms in the world, but three of them dominate the space. Google has majority share of about 80%, followed by Yahoo! with 9.5% and Microsoft’s Bing with 8.5%. Others include Ask, AOL, Baidu (63% in China) and Yandex (60% in Russia). Search engines have two main functions, namely; crawling and indexing and data mining. Before I get into the details of these functions, let me take some time to define briefly what search engines are. A search engine is a computer program designed to maintain real-time web information by running a complex algorithm on a web crawler. Crawling and Indexing The web is an interconnected links of documents in various formats including HTML, PDFs, video and graphic and media files. These links are conceptually represented on the web as nodes. A major component of all search engines is the crawler, also known as the robots or spiders. The function of the crawler is to run through all the documents on the web using the links/nodes as its pathway to finding the documents. While running through these documents, the crawler analyses and extracts content in the documents. Usually, the titles, body content and headings are extracted. Data extracted are stored and indexed in a database for quick access. In terms of storage, different search engines have different ways of doing it. While others will store pieces of the content, others may store every word in the content. For example, Google uses caching approach to store parts of the document. Data mining Once the documents are stored in the database, the next thing is to allow users to have access to them through a web interface. When a user types in a search query, the search engine runs through billions of documents and returns results after calculating relevancy and importance, using complex mathematical algorithms. After determining how relevant the query is, the program then ranks the documents in terms of their usefulness. Since there are billions of documents on the internet, one would think these processes would take many hours to complete. Fortunately, each of these processes takes a fraction of a second. Over the years, search engine developers have improved their mathematical techniques for faster processing. Not only that, computer hardwares for physical storage of data and processing of search queries have also being improved tremendously. Search companies such as Google have their own data centers across the world that house computer servers with huge storage capacities and incredible processing powers. In my next article, I will be discussing SEO and how it can search marketers influence rankings in search engines.