The site is blocking you. I just rebuilt the entire engine from scratch and saved off the test html and similar sites is returning just this
Notice this part
Its blocked. So I tried some different user agent, but that didn't do it. So you can monkey around with the header data and user agent and see if you can get it to work, but otherwise, they may just simply have a good enough blocking system that its not going to work.
Code:
<!DOCTYPE html>
<html>
<head>
<META NAME="ROBOTS" CONTENT="NOINDEX, NOFOLLOW">
<meta http-equiv="cache-control" content="max-age=0" />
<meta http-equiv="cache-control" content="no-cache" />
<meta http-equiv="expires" content="0" />
<meta http-equiv="expires" content="Tue, 01 Jan 1980 1:00:00 GMT" />
<meta http-equiv="pragma" content="no-cache" />
<meta http-equiv="refresh" content="10; url=/site/cloud" />
<script type="text/javascript">
(function(window){
try {
if (typeof sessionStorage !== 'undefined'){
sessionStorage.setItem('distil_referrer', document.referrer);
}
} catch (e){}
})(window);
</script>
<script type="text/javascript" src="/smlrdstil.js" defer></script><style type="text/css">#d__fFH{position:absolute;top:-5000px;left:-5000px}#d__fF{font-family:serif;font-size:200px;visibility:hidden}#bvbfavdddwv{display:none!important}</style></head>
<body>
<div id="distilIdentificationBlock"> </div>
</body>
</html>
Code:
<div id="distilIdentificationBlock"> </div>
Its blocked. So I tried some different user agent, but that didn't do it. So you can monkey around with the header data and user agent and see if you can get it to work, but otherwise, they may just simply have a good enough blocking system that its not going to work.