Web Spider, Web Crawler, Online Crawler, Web Site Crawler
GLOSSARY
The Trendiction Web Spider is a fully automated web crawler that regularly spiders online news articles, news comments, forums and blogs. The web spider has been developed completely in-house. We are using proprietary spidering technology to specifically spider online news and social media sites. During the spidering respectively crawling process we are extracting various entities out of the HTML page found as the main text, published date, author, title...Furthermore, we are calculating different types of metadata.