The Blueprint Training

Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
multi-threaded custom search engine problem
#1
Photo 
I am trying to create custom search engine and everything seems to work fine when I test it using "Test Engine"

also when I select "Use Detailed Harvester" everything works as expected  all 10 pages being scraped
but when I check "Use Custom Harvester" only one page seems being scraped every time I try and I am trying to figure out what am I doing wrong

here is my config from engines.dat
Code:
[Engine37]
Displayname=Seznam
QueryString=https://search.seznam.cz/?q={KEYWORD}&count=10&from={PAGENUM}
MustBeInTag=
MustNotBeInTag=
MustBeInLink=http
MustNotBeInLink=
JustBeforeLink=class="Result-title-link" href="
RightAfterLink="
PageStart=1
PageInc=10
NextPageMarker=title="Další strana"
Translation=%3f=?|%3d==|%26=&|%3a=:|&=&|%2F=/
headerData=Cookie:
Referer=0
SSLVersion=2
Favicon=/9j/4AAQSkZJRgABAQAAAQABAAD/2wBDAAMCAgMCAgMDAwMEAwMEBQgFBQQEBQoHBwYIDAoMDAsKCwsNDhIQDQ4RDgsLEBYQERMUFRUVDA8XGBYUGBIUFRT/2wBDAQMEBAUEBQkFBQkUDQsNFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBT/wAARCAAQABADASIAAhEBAxEB/8QAHwAAAQUBAQEBAQEAAAAAAAAAAAECAwQFBgcICQoL/8QAtRAAAgEDAwIEAwUFBAQAAAF9AQIDAAQRBRIhMUEGE1FhByJxFDKBkaEII0KxwRVS0fAkM2JyggkKFhcYGRolJicoKSo0NTY3ODk6Q0RFRkdISUpTVFVWV1hZWmNkZWZnaGlqc3R1dnd4eXqDhIWGh4iJipKTlJWWl5iZmqKjpKWmp6ipqrKztLW2t7i5usLDxMXGx8jJytLT1NXW19jZ2uHi4+Tl5ufo6erx8vP09fb3+Pn6/8QAHwEAAwEBAQEBAQEBAQAAAAAAAAECAwQFBgcICQoL/8QAtREAAgECBAQDBAcFBAQAAQJ3AAECAxEEBSExBhJBUQdhcRMiMoEIFEKRobHBCSMzUvAVYnLRChYkNOEl8RcYGRomJygpKjU2Nzg5OkNERUZHSElKU1RVVldYWVpjZGVmZ2hpanN0dXZ3eHl6goOEhYaHiImKkpOUlZaXmJmaoqOkpaanqKmqsrO0tba3uLm6wsPExcbHyMnK0tPU1dbX2Nna4uPk5ebn6Onq8vP09fb3+Pn6/9oADAMBAAIRAxEAPwD6G/bX+LfxT8C+J7ax0Oa68P8AhGSBNmq2UfM8pzuDS4JQjoFGCevOeOp/Y78L31pqeseIb74p2vjia/tkiksba8kuDGwbcHcy4cFdxAG0D5zXuPir4keEdNabStSuBqcz5il06zspL9yccq0cSPj/AIEK8IsvhAdf+MnhXxV4C8C33w607T7vztU1G+22gvocjMKWYYkbhkFmCYznBIFeBOk4Yr2ylz67Xen3affY/XcLmFLE5E8tqUFhvdd6nLG1S2qu5WldtWvByd3tbb//2Q==
GracePeriod=0
UserAgent=Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:56.0) Gecko/20100101 Firefox/56.0
Selected=1
DetailedSelected=1
FollowRelocation=1
AddRelative=
AddFieldValue=
ClearCookies=0
ListMode=0
ReadOnly=0

[Image: VhatEyG.png]
[Image: ZXNUtvn.png]
Reply
#2
That seems odd. Ill look at this and mail support if need be and post back.
Reply
#3
Had to mail support and they said they have made some changes in the next version so both harvesters will be identical so it should sort this. I do not have an eta as I know they are putting other things into the next update as well.
Reply
#4
Thanks for help. I will wait for next update.
Reply
#5
Your welcome, have a great day!
Reply




Users browsing this thread: 1 Guest(s)

Looplines Scrapebox List