Scalable web scraping

Hi. I have experience in scraping billions pages from instagram, twitter and facebook and saving its data to database. I can prepare some sample script to check scraping limits on this site. But I need example of inputs. Then I can prepare more accurate estimate for proxies, servers and other additional costs. Do you need some frontend ui to manage this system? Glad to discuss any details in chat. Thanks anyway.

Dear Employer, I understand your job and I will do every aspect of what you need. Please check my reviews and just give me a chance. Thanks and Regards, Ash

Hello, Dear How are you? I have read your project description and am ready for discussing with you. I have experienced in developing .NET, Java, Python, Software Architecture, Web Scraping. I will work very hard and best for you. Best Regards

Hi. I did check your project description and I think I can help you. I have experience with such a systems, check my profile for more detail. Few questions for now: 1. Do we need a UI for this or could be cmd line? 2. Yes, we will require some proxies and luminati will be the best. 3. Not sure if it's possible to reach the 250k queries per day, that is 1 query in 3 seconds. This is a search query on the website and if we are making so many then it will get slow down. but more tests will confirm this. 4. You mentioned that you need to stop/resume, is this per each query or for input file(which will have multiple queries)? 5. How fast you need this done? If you need faster then let me know and we can discuss about this. 6. Yes, anticaptcha is needed, will run some tests to see if we can pass it without any issues. 7. Get in touch with me and we can discuss more because this form is limited in chars :)) Thx, waiting for details and hope to collaborate, George.

Hi there,I am Miljan,Web Scraping expert from Bosnia & Herzegovina,Europe. I have carefully gone through with your requirements and I would like to help you with this job ! I can start immediately and finish it within the agreed deadline. Check out my profile, portfolio and former clients feedback - that'll let you know everything about me. Please feel free to contact me so that we can discuss further details. Thank you for taking the time to read my proposal.I am looking forward to hearing from you. Best regards, Miljan

Hi. I am very interested in your project, because I have much experience in such projects. I have good skills with the program language including C/C++, C#, java, php, asp.net, python, VB.NET. So I have expert and smart technic with web scraping (1000 over projects for scraping), automation and bot(500 over projects for automation and bot), selenium, mobile (android, ios native and others). Especially, I am an expert in betting project and gambling games and sites(bet365, bwin, pinnacle, betfair...) ,,, shopping auto buyer(adidas, footlocker, eastbay, supreme and etc) So, I can solve the captcha automatically. Please contact me, then we can discuss about the details and the price. And I am free in recent days, so I can do my best for your project. Thanks.

Designed and developed many web scrappers and crawlers. I also have scrapped Ali Express and daraz.pk. Contact me with more details. I am interested. Thanks

Hello, I am a Scalable web scraping expert and have plenty of good experience of it. I read your project description posted. However, I'd like to discuss more detail with you now. I am free to work now. Kindly send me a chat..,,. Looking forward your quick contacting. Regards

Hi there, I have read the details I am experienced with .NET, Java, Python, Software Architecture, Web Scraping. I can help you with this job, Please come to chat so we can discuss this job.

Hi! I am an experienced .net software developer (+9 years). I am interested in this project and would like to discuss with you some details. I already have previous experience with webscraping tools.

Hi there. I am expert in web scraping and have more than 7 years of experience. I can do any kind of web scraping. I have read your job details and i am glad to tell you that i can do this job for you. Please feel free to contact me to discuss further about the task. looking forward to hearing back from you. Thanks

Hello Sir, We are expert in web scrapping using java and selenium. Please give us a chance to do a great job for you. Please ping me we can discuss on your requirement and can start immediately. Thanks

How about a Chrome extension? UI to upload and manage everything you have listed. Run on any OS and just switch IP to use it. Quick Chat?? Thanks, Satya - Code with Care | 100% Project Completion Rate

Hi sir, i have read your requirement carefully. I can do scalable web scraping perfectly and exactly without any risk. If you wanna confirm me, please check my work history and portfolio kindly. Then, you can know about my ability that can do your project enough. Especially, this is my experienced and successful work before. Also i have fluent English skill, so you can communicate with me very easily via voice call and thus, you can save your time. No problem for working with me now. So if we work and do our best for one purpose, we can get great success each other. Please call me and discuss about your project seriously, then you can be at ease about your project. I'm always here and ready to talk with you right now. I'll wait for your call. Thanks.

Hi. Web scraping is my speciality. I've been doing it for many years now. Send me a message so we can talk about it.

Can build this scraping tool. Can provide option for stop start and pause for a running job. Can use luminati proxy. Have done this kind of job before as well. these are my skills related to web scraping Have done scraping in CasperJS Phantomjs, python. Have done testing and automation with selenium also. Know to deal with database like mongo, mysql, Elasticsearch. Also know to handle proxy and captcha while scrapping

Hello there! I have checked the website and i can do this using python and selenium. Selenium hast to be used because it has recaptcha which cannot be done by using some request or scrapy modules of python. Anti-captcha charges aroudn 1.8 $ for 1000 captchas solved and i have used it before as well. We would need proxies as well to disguise our IP and avoid getting blocked. I am not sure if we can do 250k requests a day, cause it depends on the speed and capacity of the system you are using for how many threads it can run at once of selenium. please have a look at my reviews and ping me! My skills & experience: -- 2.8 years of experience in building automation tools using python. -- Worked for Infosys ( a multinational company ) as Automation engineer. -- Worked on ATP world tour 2017 project as Data analyst. -- Can scrape content of sites protected with captchas and have dynamic content generated using ajax and JS. -- Built more than 56 web scraping projects till date ( 25 on freelancer.com, please check reviews ) -- Websites i have built scraping tools for include Linkedin, Facebook, Gmail, Tinder, Okcupid, Outlook and many more. -- Can bypass websites protected with captchas ( reCaptcha v2 and invisible Captcha ). Looking forward to hear from you. Thank you, have a nice day!

With respect to this project I would like to present myself as a candidate for your consideration. I have more than 8 years of IT experience. I have successfully completed few projects which involved Programming, scripting, web scraping. Key features of my work are as follows: 1) Always within schedule 2) Always within scope Looking forward to hear from you. Regards

hello, it will be a webpage in front(ptthon3 + Flask) + a MQ in the middle(rabbit mq) + many tasks in the backend (can be 1 or many ). mostly it will use python3 please check my profile for details..

Ready to start the work to make the scrapper for your requirement to get the data from other websites,please check my recent similar work ,we can discuss more over chat,thanks regards Arjun S. Scrapping projects link : https://www.freelancer.com/projects/python/Data-Scraping-Real-Time-for/ https://www.freelancer.com/projects/php/web-scrap-betradar/