Skip to Content

New Blog Post: How to keep all your websites in sync with scraping technology

Technology Blog

Technology Blog

Articles by Lisa King

Tuning Site Search for Covid-19

The recent spike in usage of the term covid-19 introduced some inconsistent results in Imaginary’s custom site search engine, iScraper, a tool which utilizes Elasticsearch as its indexing engine.  We’ll take a closer look at these results and show how we corrected the problem with an Elasticsearch configuration change. iScraper...

Remember your robots.txt

Because the development site is also on the web, it needs its own web address (e.g. dev.yourcompany.com).  There are many adjustments needed in the configuration file to accommodate the development URL, especially when testing third-party connections.  For example, ecommerce gateways don't like when credit card information is coming from a...

Back