The Rise of Smart Crawlers