PhantomJS fails to get content from specific website


#1

So i have code that opens pages and loads their content, and it works for all other websites, but when i try it on http://www.wayfair.com/ , it always gets redirected to a sorry page, the same thing happens when i try to get the page content through node curl, my guess is http://www.wayfair.com/ have some way to detect requests not coming from humans, i could be wrong though. Is there a way to get around this and what’s really the issue here.


#2

Did you try to set the user agent?
var webpage = require(‘webpage’).create();
webpage.settings.userAgent = ‘Mozilla/5.0 (X11; FreeBSD i386; rv:49.0) Gecko/20100101 Firefox/49.0’