
Closed
Posted
Paid on delivery
I already have a Python-based Scrapy project and now need dedicated spiders that can harvest data from the mobile apps Rabbitmart (Egypt), Voo (Egypt) and Oscar (Egypt). The crawlers should pull every publicly available endpoint (or reverse-engineered one) required to deliver: product details, user reviews, pricing information, the underlying product IDs, any active promotions or discounts and—where the apps expose it—current stock levels. All captured information must be written to clean, well-structured JSON files so that I can feed it straight into my existing pipeline. I’m still unsure whether the apps will demand login or token-based authentication; therefore, please build the spider with the flexibility to plug in credentials, headers or session cookies if we discover they’re needed while testing. Deliverables • A fully-commented Scrapy spider (or set of spiders) that crawls Rabbitmart, Voo and Oscar • Example JSON output for each app covering every requested field • A short README explaining setup, any extra libraries (e.g. mitmproxy, adb, Playwright) and how to toggle authentication if required Acceptance criteria Data for at least 100 products per app, matching the requested fields, can be reproduced on my machine with one Scrapy crawl command.
Project ID: 40581930
76 proposals
Remote project
Active 1 day ago
Set your budget and timeframe
Get paid for your work
Outline your proposal
It's free to sign up and bid on jobs
76 freelancers are bidding on average $141 USD for this job

⭐⭐⭐⭐⭐ Create Dedicated Scrapy Spiders for Mobile Apps Data Harvesting ❇️ Hi My Friend, I hope you're doing well. I've gone through your project needs and see you're looking for dedicated spiders to harvest data from Rabbitmart, Voo, and Oscar in Egypt. You don't need to look any further; Zohaib is here to help you! My team has successfully handled over 50 similar projects for data scraping. I will create efficient spiders to gather all required data and ensure it's structured perfectly for your pipeline. ➡️ Why Me? I can easily build your dedicated Scrapy spiders as I have 5 years of experience in web scraping and data extraction. My expertise includes Python programming, data structuring, and API integration. I also have a strong grip on reverse engineering and handling various authentication methods, ensuring a smooth process for your project. ➡️ Let's have a quick chat to discuss your project in detail and let me show you samples of my previous work. Looking forward to discussing this with you in chat. ➡️ Skills & Experience: ✅ Python Programming ✅ Scrapy Framework ✅ Data Extraction ✅ JSON Structuring ✅ API Integration ✅ Web Crawling ✅ Data Cleaning ✅ Authentication Handling ✅ Reverse Engineering ✅ Error Handling ✅ Data Analysis ✅ Documentation Writing Waiting for your response! Best Regards, Zohaib
$150 USD in 2 days
8.1
8.1

With extensive experience in web scraping using Python and Scrapy, I propose creating dedicated spiders for Rabbitmart, Voo, and Oscar mobile apps. These spiders will extract product details, user reviews, pricing, product IDs, promotions, and stock levels into structured JSON files. The spiders will handle authentication requirements flexibly and ensure data accuracy and completeness. A comprehensive README will be provided for setup and integration. Output will include a minimum of 100 products per app, supporting your data needs effectively.
$225 USD in 5 days
7.1
7.1

⭐⭐⭐⭐⭐ Scrapy Spider For Mobile Apps -- 2 ❇️ Hi. Ive reviewed your project needs and see youre looking for dedicated Scrapy spiders to harvest data from mobile apps Rabbitmart, Voo, and Oscar in Egypt. I have considerable expertise in developing Scrapy projects specifically tailored for data extraction from mobile applications. I will create efficient and robust spiders capable of fetching and organizing data into clean JSON formats, which will easily integrate with your existing pipeline. ➡️ Why Me? I hold a PhD in Computer Science with a focus on data scraping technologies and have more than 8 years of experience in web and mobile app scraping. My expertise includes Python programming, reverse engineering of APIs, and handling data extraction under various authentication scenarios. I am also skilled in using additional tools like mitmproxy and Playwright which are often necessary for mobile app data extraction. ➡️ Let’s have a quick chat to discuss your project in detail, and I can show you samples of my previous work. ➡️ Some of my similar work: ✅ E-commerce product data scraping ✅ Social media analytics tools ✅ Custom analytics for mobile app usage ✅ Real-time stock level scraping tools ✅ Pricing comparison systems ✅ User review aggregation tools ✅ Authentication-enabled data scrapers Waiting for your response! Best Regards, Dr. Muhammad Asad
$200 USD in 6 days
6.8
6.8

Hello, I HAVE DEVELOPED SIMILAR SCRAPY SPIDERS, MOBILE APP SCRAPERS, AND API REVERSE-ENGINEERING SOLUTIONS BEFORE AND CAN SHARE RELEVANT DEMOS OF MY PREVIOUS WORK. >>>> Multi languages (English and Arabic)Left-To-Right (LTR) and Right-To-Left (RTL) <<<< I have carefully reviewed your requirements and fully understand your existing Scrapy project. I can build dedicated spiders for Rabbitmart, Voo, and Oscar to extract product details, reviews, pricing, product IDs, promotions, discounts, and stock availability where exposed. The solution will support authentication using tokens, headers, or session cookies if required and generate clean, structured JSON output compatible with your existing pipeline. I HAVE 10+ YEARS OF EXPERIENCE IN PYTHON, SCRAPY, PLAYWRIGHT, API REVERSE ENGINEERING, MOBILE APP DATA EXTRACTION, JSON PROCESSING, AND AUTOMATION. I follow an Agile development approach with clean, well-documented code, reliable extraction logic, and comprehensive testing. I WILL PROVIDE 2 YEARS OF FREE ONGOING SUPPORT, COMPLETE SOURCE CODE, SAMPLE JSON OUTPUTS, README DOCUMENTATION, AND ASSIST YOU UNTIL THE SPIDERS RUN SUCCESSFULLY IN YOUR ENVIRONMENT. I am available to start immediately and will remain committed until all deliverables are completed according to your requirements. I eagerly await your positive response. Thanks, Christina
$500 USD in 15 days
7.0
7.0

Hi, I'd handle this by building targeted Scrapy spiders with Appium integration for mobile app interaction, since direct API calls aren't always exposed. The biggest improvement will come from isolating authentication handling in middleware so we can switch between no-auth, token-based, or session-based access without rewriting core logic. When I worked on a similar project for a local delivery app, I ran into the same authentication puzzle—some endpoints required session cookies that only appeared after logging in via Playwright. The crawler still pulled 200+ products per run once we got the flow right. Changes will be modular and tested in isolation, so you can review each spider's behavior before they go live. The spiders will also include basic retry logic and user-agent rotation to avoid blocks. This approach keeps your existing pipeline intact while giving you reliable, structured data that matches your JSON schema. If the setup matches your needs, I can review the current codebase and suggest the safest path forward. Thanks, Denis.
$100 USD in 3 days
6.0
6.0

Good morning, I can help build dedicated Scrapy spiders for Rabbitmart, Voo, and Oscar apps to extract product details, user reviews, pricing, promotions, and stock levels. The spiders will output clean JSON files ready for your pipeline. I will ensure flexibility for token or session-based authentication if needed during testing. This will include fully commented code and a README with setup instructions and authentication toggles. Do you already have access to the apps or any API documentation to start reverse engineering? Best regards, Waqas & GoDesign Team
$30 USD in 3 days
5.6
5.6

Hello There! I’m Md Toriqul Islam, and I’m excited to partner with you. I can dive into your project immediately. I have rich experience in Python, Scrapy, Playwright, API integration, reverse-engineering mobile app traffic, JSON processing, and scalable web scraping. I understand you need dedicated Scrapy spiders for Rabbitmart, Voo, and Oscar to extract structured product data, pricing, reviews, promotions, IDs, and available inventory where exposed. I can build flexible, well-documented spiders with optional authentication support and clean JSON output that integrates seamlessly with your existing pipeline. I am skilled in Python, Scrapy, Playwright, mitmproxy, ADB, REST APIs, JSON, and data extraction. I’m ready to start immediately and would be happy to discuss this project. Looking forward to hearing from you. Best regards, Md Toriqul Islam
$100 USD in 3 days
5.6
5.6

Hi, I’m a senior developer with 20+ years of experience building and maintaining production-grade data extraction systems, including complex Scrapy spiders for mobile app APIs and hybrid scraping workflows, so this is a straightforward task. I’ll review the three target apps to identify their endpoints, authentication needs, and rate limits, then build three Scrapy spiders—one for each—configured to extract product details, reviews, pricing, IDs, promotions, and stock levels from every public endpoint or reverse-engineered API. The spiders will normalize responses into clean JSON and write them to disk in a single crawl, with placeholders for credentials, headers, or session tokens if login becomes necessary during testing. I’ll include a README with setup steps, dependencies (e.g., mitmproxy for traffic inspection or Playwright for headless fallback), and toggles for authentication once we confirm requirements. I can start immediately and deliver a working prototype within a day.
$150 USD in 2 days
5.3
5.3

You need someone who can reverse-engineer mobile app APIs and build Scrapy spiders that actually deliver clean, structured data from Rabbitmart, Voo, and Oscar. I've done similar work pulling data from apps that don't have public web interfaces, including handling cases where authentication turned out to be required mid-project. For each app, I'll start by intercepting traffic using mitmproxy or a similar tool to map out the available endpoints and understand what fields they expose. If the apps use token-based auth or require login, the spider will be built with a configurable auth module so you can drop in credentials or headers without rewriting code. The output will be clean JSON matching your existing pipeline's structure, and I'll include example outputs for all three apps. I'll also provide unlimited revisions until you're completely satisfied. Let me know if you'd like to discuss the approach further or jump into testing.
$140 USD in 7 days
5.2
5.2

As the founder of Solves Inn, a software development studio, I am well-versed in crafting tailor-made solutions to meet complex requirements. With a focus on scalability and efficiency, my team and I are proficient in Python, one of the main technologies required for this task. We also have extensive experience with API development and software architecture, two crucial skills needed to build the desired Scrapy Spider. The comprehensive understanding we have of mobile apps and their endpoints gives us an upper hand in reverse-engineering them to extract all available data. Moreover, our familiarity with various third-party libraries (mitmproxy,adb) and automation tools such as Playwright equips us with the necessary flexibility to adapt to any authentication changes that may arise during testing. In addition to meeting your project's primary requirements - building dedicated spiders for Rabbitmart, Voo, and Oscar - we at Solves Inn go the extra mile. We will not only provide a well-commented Scrapy spider but also offer clear instructions on setup via a detailed README file. You can expect example JSON output for each requested field as well. Further solidifying our commitment to delivering quality work is the fact that we will produce data for at least 100 products per app matching all your specified fields - sure to give you confidence in our results. Choose us, and together let's "solves in" your data harvesting needs!
$100 USD in 5 days
5.2
5.2

With my expertise in both Python and web scraping, I am confident that I can deliver exceptional results on your Scrapy spider project for the Rabbitmart, Voo, and Oscar mobile apps. Over the years, I have honed my skills to be adaptable and meticulous - two characteristics that are highly sought after for projects of this nature. By utilizing these skills, I can quickly and effectively reverse-engineer the data endpoints you need to scrape and ensure all extracted information is organized and stored in clean JSON files - ready to be fed into your pipeline. In addition to my specific experience working with Scrapy, I have adept knowledge of Python libraries like mitmproxy, adb, Playwright that may be useful in your project. In fact, Ihre Beschreibung parametriert mir auch Szenarien wie eine mögliche Login-Authentifizierung oder eine solche per Token. Hierbei profitieren Sie von meiner vielseitigen Erfahrung in der Applikationsentwicklung und Datenanalyse bis hin zu den Grundlangen der Künstlichen Intelligenz mit Anwendungen von Python. I always prioritize customer satisfaction by ensuring the final product aligns impeccably with their requirements. With this in mind, not seulement will you receive fully-commented Scrapy spiders for all three apps, but a README instruction detailing the setup process and any additional libraries used for extra transparency.
$30 USD in 7 days
6.4
6.4

I’ve already completed similar Scrapy projects and can deliver dedicated spiders to harvest data from Rabbitmart, Voo, and Oscar quickly and efficiently! ? With over 6 years of experience in Python and web scraping, I understand the nuances of extracting data from mobile applications. I'll ensure your spiders are flexible enough to handle any authentication methods needed during testing and will output well-structured JSON files for seamless integration into your existing pipeline. 1️⃣ Can you confirm if there are any specific data fields beyond what you've listed? 2️⃣ Are there any particular performance benchmarks or constraints I should be aware of? 3️⃣ Would you like me to include any specific error handling in the spider? Let’s connect — just click the chat button, and let’s get your work done quickly. I’m ready and waiting for your message.
$200 USD in 10 days
4.7
4.7

Nice to meet you , My name is Anthony Muñoz, I express my interest in working on your project after carefully reading the requirements and concluding that they match my area of knowledge and skills. I am currently the lead engineer for the IT agency DSPro and I have more than 10 years of experience in the field. I have successfully completed a large number of similar jobs and I consider your project to be a challenge in which I would like to work and be able to make it a reality. Please feel free to contact me, it will be my pleasure to help you. I greatly appreciate the time provided and I remain attentive to any questions or concerns. Greetings
$136 USD in 7 days
4.6
4.6

I’ve built reverse-engineered scrapers for mobile apps using Scrapy + Playwright + mitmproxy to extract product data, reviews, and pricing from closed APIs, so this is straightforward. I’ll implement three domain-specific spiders (Rabbitmart, Voo, Oscar) under your existing project, each using Playwright for browser automation to intercept endpoints and mitmproxy for TLS decryption if needed. Each spider will write product IDs, details, reviews, pricing, promotions, and stock levels to structured JSON in the same schema, with a settings module for pluggable auth headers, cookies, or tokens via environment variables. The spiders will run headless but expose config flags to toggle authentication or proxy interception during testing. I’ll include sample outputs matching your pipeline schema and a README covering setup with Playwright, mitmproxy, and ADB for device control, plus env vars for credentials. I can start immediately.
$175 USD in 2 days
4.4
4.4

Hello, I’m interested in developing the Scrapy spiders for Rabbitmart, Voo, and Oscar mobile apps. I have experience building Python-based data extraction solutions, API-driven crawlers, and automation workflows. I can help analyze the mobile app data flow, identify available endpoints, and build maintainable Scrapy spiders that deliver structured JSON output for your existing pipeline. I can provide: * Fully commented Scrapy spiders for each application. * Reverse-engineering of publicly available APIs/endpoints where applicable. * Extraction of product details, IDs, pricing, reviews, promotions, and stock information when available. * Flexible authentication handling with support for headers, tokens, cookies, and session management. * Clean JSON output structures ready for downstream processing. * Setup documentation including required libraries, configuration steps, and crawl instructions. My approach includes: * Inspecting app network behavior and API requests. * Building reliable request handling, parsing, and error recovery. * Testing data quality with sample outputs. * Ensuring the spiders are easy to maintain and extend. I understand the importance of reproducibility, so I will ensure the final solution can collect the required product data with a simple Scrapy command and provide clear documentation for future use. I’d be happy to discuss the apps, expected fields, and existing Scrapy project structure before starting. Best regards, Tran Viet
$140 USD in 7 days
4.4
4.4

Hello, After reviewing your project requirements in detail, I fully understand the scope and expectations. I have experience developing Python scraping systems, API data extraction tools, and automated data pipelines, and I’m available to start your project immediately. I bring strong expertise in Python, Scrapy, Web Scraping, JSON, Data Extraction, API Integration, Software Architecture, and Google App Engine with over 10 years of experience. One of the key challenges in projects like this is extracting structured data from mobile applications where endpoints may require reverse engineering, authentication tokens, or dynamic headers. I can analyze the app communication flow, build flexible Scrapy spiders, handle authentication when required, and deliver clean JSON outputs containing products, reviews, pricing, promotions, IDs, and stock information. I have experience creating maintainable crawlers with proper documentation, reusable architecture, and data formats ready for existing pipelines. I have a couple of quick questions: • Do you have any existing API captures, network logs, or app access available for Rabbitmart, Voo, and Oscar? • Should the spiders only collect current data, or also track price/stock changes over time? I would be glad to help build reliable Scrapy spiders and provide reproducible results with complete documentation. Best regards, Carlos.
$30 USD in 7 days
4.1
4.1

Drawing on my extensive experience in full-stack web development and software engineering, I bring a tailor-made perspective to your data harvesting needs. With expertise in Python-based Scrapy and a proven track record of building scalable, high-performance applications, I am uniquely positioned to deliver a fully-commented Scrapy spider that will not only scrape the required endpoints but do so with the necessary flexibility to handle potential login or token-based authentication. My commitment to well-structured, clean JSON output means that you can seamlessly feed the harvested data to your existing pipeline for immediate use. I will further provide you with an example JSON output for each app, covering every requested field, as well as equip you with a clear README outlining setup procedures, necessary libraries (such as mitmproxy, adb, Playwright) and instructions on plugging in authentication if required. Ultimately, my aim is to produce results that matter for your business. My dedication to quality is exemplified by setting an acceptance criterion of delivering data for at least 100 products per app and ensuring they match the requested fields and can be reproduced on your machine with just one Scrapy crawl command. Choose me today so that we can get started on optimizing your data harvesting initiatives
$449 USD in 7 days
3.7
3.7

With my extensive experience in web scraping and the proficiency I possess working with both Google App Engine and Python's Scrapy framework, I believe I'm an ideal fit for your project. I'm quite adept at navigating complex data architectures and understand the importance of scalable and clean code that aligns with your mandate. I am positive that I can deliver robust spiders to scrape data from Rabbitmart, Voo, and Oscar, even if there are complexities such as logins or token-based authentication. My familiarity with managing headers, sessions, and credentials using Scrapy will ensure the spiders remain fluid during testing. Rest assured that you can expect a well-documented Scrapy spider optimized for your requirements, extracting crucial data like product details, user reviews, pricing information, stock levels, underlying product IDs, promotions/discounts (if applicable). I'll also furnish you with clear, structured JSON output as needed along with a useful README to streamline navigation."
$120 USD in 2 days
3.9
3.9

Hello, I can deliver a Scrapy spider for Rabbitmart, Voo, and Oscar, harvesting product details, reviews, pricing, promotions, and stock levels. I'll ensure clean JSON output, flexible authentication, and a clear README for setup. With 5+ years of experience, I'll reverse-engineer endpoints, handle authentication if needed, and meet your acceptance criteria. Send a message to discuss further or see samples of similar projects. Thanks, Adegoke. M
$150 USD in 3 days
3.8
3.8

Dear Sir, I am thrilled to bid your project. I can extend your existing Scrapy codebase with dedicated spiders for Rabbitmart, Voo, and Oscar, producing clean JSON outputs for product IDs, titles, descriptions, reviews, pricing, promotions, discounts, and available stock data. I will first inspect each app’s public network traffic and API behavior, then implement reusable request handling, pagination, retries, throttling, headers, tokens, cookies, and optional authentication configuration without hard-coding credentials. Where standard Scrapy requests are sufficient, I will keep the solution lightweight; Playwright, mitmproxy, or ADB-based inspection will only be used where technically necessary and permitted. Each spider will include clear field mapping, duplicate prevention, error logging, response validation, and reproducible commands for collecting at least 100 products per app. You will receive fully commented source code, sample JSON files, dependency details, and a concise README explaining setup and authentication options. One crucial question: do you already have valid test accounts and permission to inspect authenticated traffic for these apps if public endpoints do not expose reviews, promotions, or stock levels? Sincerely, Adison.
$140 USD in 7 days
3.8
3.8

Cairo, Egypt
Member since Jul 14, 2026
$30-250 USD
$30-250 USD
$200-600 USD
$250-750 USD
$30-250 USD
$30-250 USD
$30-250 USD
$15-25 USD / hour
$30-250 USD
$250-750 USD
₹600-1500 INR
$250-750 USD
£3000-5000 GBP
$30-250 USD
£10-15 GBP / hour
₹750-1250 INR / hour
€250-750 EUR
$30-250 USD
₹1500-12500 INR
$10-30 USD