
Closed
Posted
Paid on delivery
UPDATED - Build a reliable, locally runnable "Coles & Woolworths Price Data Extraction Tool" that collects weekly Coles and Woolworths product pricing data across Australia. - (UPDATED) Build a tool (Use TypeScript on Node.js) that can be executed automatically every week. - Scrape product data from Coles and Woolworths websites for all available store locations across Australia. - Extract the following fields: product name, package size/weight/volume, current price, unit price (e.g. $/kg), retailer, store location, product URL, promotional price (if applicable), and extraction date. - Capture location-specific pricing, including regional price differences and store-specific promotions, wherever available. - Collect all available products, including different sizes and variants, while avoiding duplicate records. - Export the extracted data in a structured format such as CSV or JSON. - Include logging, error handling, retries, and a summary of successful and failed extractions and uploads. - Make retailer settings, store locations, execution schedule, and AWS configuration configurable without changing the core code.
Project ID: 40759991
278 proposals
Remote project
Active 4 hours ago
Set your budget and timeframe
Get paid for your work
Outline your proposal
It's free to sign up and bid on jobs
278 freelancers are bidding on average $432 AUD for this job

Hi, The tricky part is capturing prices for the same product across different Coles and Woolworths stores without missing variants or creating duplicate records. The tool needs to handle regional prices and promotions reliably every week. We’ll build this in TypeScript on Node.js, with configurable retailer and store settings, scheduled extraction, and structured CSV/JSON exports. First, we’ll check how both websites expose product and location-specific pricing. Then we’ll build the extraction, variant handling, duplicate checks, retries, and error logs. Finally, we’ll configure AWS options, execution scheduling, upload summaries, and handover documentation. Before starting the work, I want to clarify: 1. Do you need pricing from every store in Australia, or only stores where location-specific prices are publicly available? 2. Where should the weekly CSV/JSON files be stored, and will you use the data for price comparisons, market research, or another purpose? We can confirm the extraction approach and key limitations before finalising the scope. Best, Sandeep
$500 AUD in 7 days
9.1
9.1

As a renowned data aficionado, my goal is to transform your repetitious data processes into powerful, automated workflows. Powered by Python and the titan AWS, I'm more than equipped to create your Coles & Woolworths Price Data Extraction Tool. With a realm of experience specifically in dynamic web scraping like yours, whether it’s Scrapy or BeautifulSoup –I’ve got you covered. My expertise allows me to extract not only the standard fields such as product name and price, but also niche attributes like package size and weight to give you a comprehensive dataset. When it comes to executing the scraper automatically while attending to location-specific pricing variations and avoiding duplicates, I’ve implemented highly successful methods in past projects. You mentioned that you’d prefer a structured file format such as CSV or JSON for easy data understanding; this is precisely how I think too. Rest assured, error handling, retries, logging – all aspects of fault-tolerance will be taken care of with meticulous precision. In the realm of data work, dedication and accuracy hold utmost value; my clients speak highly of my own commitment to these attributes. My skills scale from B2B lead generation to eCommerce Store Operations, making me a versatile and well-rounded candidate who can optimize your workflow and scale your operations. I am ready when you are— let's create a smarter solution together!
$250 AUD in 1 day
8.9
8.9

╰┈➤ PYTHON DATA EXTRACTION DEVELOPER | RETAIL PRICE MONITORING Hi there, ✔️ I can build a locally runnable Python tool to collect Coles and Woolworths product pricing data, organise it by retailer and location, and export clean datasets for weekly analysis. Key Features: ✦ Product names, sizes, prices and unit pricing ✦ Promotional prices, product URLs and extraction dates ✦ Configurable store locations and retailer settings ✦ Product variants and duplicate detection ✦ CSV/JSON exports with consistent data structure ✦ Weekly scheduling and AWS configuration ✦ Logging, retries and extraction failure reports ✦ Summary of successful and failed runs or uploads ❓ Do you have preferred store locations or retailer accounts for accessing location-specific prices? ❓ Should the weekly data be saved locally, uploaded to AWS S3, or both? ⚡ I’ll keep the tool modular, configurable and easy to run weekly, with clear setup instructions and an auditable extraction report. ➰ Looking forward to discussing the requirements! Best regards, Team DDS.
$500 AUD in 7 days
8.7
8.7

I can build a weekly Python scraper for Coles and Woolworths with store-level prices, CSV/JSON exports, AWS support and error logs. Sample ready.
$250 AUD in 2 days
8.3
8.3

Hi, This is Elias from Miami. I checked your project description and understand you need a locally runnable, scheduled price-extraction tool covering Coles and Woolworths products across Australian locations, including store-specific pricing, promotions, unit prices, URLs, and extraction dates. I’ve worked with Python scraping, structured data pipelines, scheduled jobs, AWS, retries, logging, deduplication, and export workflows. I’d build the scraper with retailer-specific adapters, configurable store/location settings, robust request handling, and a normalized output schema. The weekly process would include retries, validation, duplicate prevention, structured CSV/JSON exports, and a clear success/failure summary. I’d also keep retailer logic and AWS/scheduling configuration separate from the core extraction engine so future changes can be made without rewriting the application. I have a few questions for clarification: Q1 – Do you already have approved access/API credentials for either retailer, or should the tool rely solely on publicly accessible website data? Q2 – How should the system represent products that have different prices across multiple stores or regions? Q3 – Should weekly runs be stored historically in separate files, or accumulated into a single dataset/database for price comparison? Looking forward to hearing from you.
$500 AUD in 7 days
8.3
8.3

⭐⭐⭐⭐⭐ Build a Weekly Coles & Woolworths Price Scraper Tool ❇️ Hi My Friend, I hope you are doing well. I checked all of your project requirements and I can see you are looking for a price data extraction tool. You have no need to look any further as Zohaib is here to help you! My team is already doing 50+ similar projects for web scraping and data pipelines. I will build a Python script that runs weekly, scrapes both sites, saves clean CSV/JSON and logs all actions. I will also add config files for stores, schedule and AWS so you can change settings without touching code. ➡️ Why Me? I can easily do your scraper as I have 5 years of experience in Python, Selenium, BeautifulSoup, AWS, cron jobs, data cleaning. Not only this, I have a strong grip on REST APIs, JSON handling and error recovery. ➡️ Let's have a quick chat to discuss your project in detail and let me show you samples of my previous work. Looking forward to discussing with you in chat. ➡️ Skills & Experience: ✅ Python ✅ Selenium ✅ BeautifulSoup ✅ Scrapy ✅ AWS S3 ✅ Lambda ✅ Cron Scheduling ✅ CSV Export ✅ JSON Export ✅ Logging ✅ Error Handling ✅ Data Cleaning Waiting for your response! Best Regards, Zohaib
$350 AUD in 2 days
8.0
8.0

Hi, I understand you need a locally runnable Python tool that collects Coles and Woolworths product prices across Australian store locations every week. I will build the extraction flow around retailer APIs or stable website endpoints where available, with location-based pricing, variants, promotions, unit prices, product URLs, and extraction dates. The Software Architecture will keep retailers, locations, schedules, and AWS settings outside the core code, so updates are simple. Automation will include retries, logging, duplicate prevention, export to CSV or JSON, and a clear success/failure summary. I will also make the API handling and error recovery easy to test and maintain. Should the tool use official retailer APIs where available, or should I design browser-based extraction for location-specific prices and promotions? Please start the chat and let’s discuss the store coverage, output format, and preferred AWS setup.
$650 AUD in 10 days
7.3
7.3

Hello, I can develop a reliable, locally runnable Python tool to collect weekly product pricing data from Coles and Woolworths across Australia. It will capture product details, package sizes, current and promotional prices, unit prices, store locations, and product URLs while minimizing duplicates and handling regional price differences. I’ll include automated scheduling, configurable retailer and store settings, CSV/JSON exports, robust logging, retries, error handling, and extraction summaries. The architecture will also support configurable AWS integration without modifying the core code. My focus will be on building a maintainable, scalable, and well-documented solution that runs reliably with minimal manual intervention. I’m ready to get started.
$350 AUD in 5 days
7.3
7.3

Hi, The main challenge is scope. Collecting weekly prices for every product across nearly 2,000 Coles and Woolworths stores could mean tens of millions of records. I'd clarify the requirements first to avoid building the wrong solution. Three quick questions: 1. Store coverage: Every store or a representative sample by region? 2. Product coverage: Full catalogue or selected categories? 3. Deployment: Run locally on a schedule or on AWS with results saved to S3? Both retailers expose product data through internal JSON endpoints, which are faster and cleaner than parsing HTML. Bot protection means the scraper also needs proper session handling, retries, and request pacing. I've built scheduled pipelines collecting 400,000+ public registry records across all 50 US states, with deduplication, error handling, and run reports. I'd start with one retailer and a small store sample, delivering extraction, CSV/JSON exports, logging, and a run summary before scaling up. Best, Tuan N.
$480 AUD in 7 days
7.3
7.3

Youssef, Full-Time Python Developer specializing in reliable, large-scale retail price scraping. You need a locally runnable tool for weekly Coles and Woolworths price data across all Australian store locations. My approach uses Playwright to handle dynamic site content and anti-bot measures, ensuring accurate extraction of location-specific pricing, promotions, and unit prices. The scraper will be built as a configurable Python package with modular selectors for each retailer. I'll implement robust logging, retry logic for failed requests, and deduplication to clean the data before exporting to CSV or JSON. Scheduling will be handled via a separate cron script or AWS Lambda configuration, keeping the core code separate from your AWS settings. I've completed multiple similar projects for Australian grocery price tracking. To confirm: do you have a preferred method for storing the weekly output files, such as an S3 bucket path? Ready to start immediately.
$750 AUD in 1 day
7.4
7.4

I'd first verify how Coles and Woolworths select stores, so weekly runs capture local prices and promotions rather than repeating national prices across Australia. I'll build Python collectors that discover available stores, paginate products and variants, and deduplicate by retailer, store and product ID. CSV/JSON exports will include name, pack size, current/unit/promotional prices, retailer, location, URL and extraction date, with unavailable local pricing flagged. Weekly scheduling, retries, logs and extraction/upload summaries will be included; retailer, store, schedule and AWS settings will stay outside the core code. You'll receive the locally runnable tool, source and configuration in 5 days, with an initial nationwide run to check coverage. Should weekly exports also upload to an AWS S3 bucket, or stay local? Muhammad Saad
$250 AUD in 5 days
6.8
6.8

Hi! This is a project we can take on confidently — large-scale retail scraping with recurring runs is territory we're comfortable in. Before locking down the approach, one thing matters a lot: do you need pricing from every physical store location, or is national/regional online pricing enough? Coles and Woolworths expose per-store pricing only once a store or postcode is selected on their site, so covering every location means simulating that selection for each one — which multiplies the run time and the complexity of avoiding duplicate or stale records. If regional-level pricing covers your needs, the build is considerably simpler and faster to run each week. On the approach: we'd use Python with Scrapy or Playwright depending on how much each site relies on JavaScript rendering for product listings — Woolworths tends to need rendering, Coles less so. Retailer settings, store lists, schedule, and AWS config would live in external config files, not hardcoded, so you can adjust coverage without touching the core script. Logging and retry logic would track failed extractions per store so a single blocked request doesn't silently drop data. Output would be clean CSV/JSON with consistent field names across both retailers, with a run summary attached. Weekly execution via cron or a scheduled AWS job, and recurring hosting costs would run separately from the build itself. I'd like to go over the store-coverage question before finalizing scope. Gustavo & the DoTheCode team
$450 AUD in 7 days
6.9
6.9

Dear client, I can do this job of Coles & Woolworths Price Data Extraction Tool accurately as per your requirements and available to start immediately. Thanks!
$750 AUD in 7 days
7.0
7.0

Hello I checked your project and the main point is to build a locally runnable Python scraper for weekly Coles and Woolworths pricing, including store-level differences, promotions, product variants, and AWS-ready exports. In the past I built similar data collection systems using Python, Scrapy, Selenium, Puppeteer, and Ads Power, including large Google Maps datasets with duplicate control and structured processing. I would use a suitable approach for each retailer, since their location and product data may be loaded dynamically. I would build configurable retailer and store settings, product normalization, deduplication, retry handling, and detailed logs. The tool can export CSV or JSON and include an execution summary for successful, failed, and uploaded records, with AWS settings kept outside the core code. Could we have a short conversation to confirm the preferred AWS destination, local execution method, and how store locations should be discovered across Australia? This will help me choose the cleanest scraping and scheduling setup. Best regards
$999 AUD in 9 days
6.6
6.6

Hello, LET’S BUILD A RELIABLE PRICE EXTRACTION TOOL THAT AUTOMATES WEEKLY PRODUCT DATA COLLECTION ACROSS AUSTRALIA. I recently developed a similar data extraction tool and can show you a live demo. With 12 years of IT experience, I can develop a Python-based solution to collect Coles and Woolworths product data across available Australian store locations, including product names, package sizes, current and unit prices, promotional prices, retailer, store location, product URLs, and extraction dates. The tool will capture store-specific pricing wherever accessible, handle different product sizes and variants, and prevent duplicate records. I’ll implement automated weekly execution, configurable retailer and store settings, CSV/JSON exports, and optional AWS integration for data uploads. The solution will include efficient data processing, logging, retries, error handling, and execution reports showing successful and failed extractions. The architecture will be modular, allowing you to update locations, schedules, and AWS settings without changing the core code. Data coverage will depend on retailer access and available location-specific pricing. You’ll receive tested source code and clear setup instructions for running and maintaining the tool locally. I WILL PROVIDE 2 YEARS OF FREE ONGOING SUPPORT AND COMPLETE SOURCE CODE. WE WILL WORK WITH AGILE METHODOLOGY AND WILL ASSIST YOU FROM ZERO TO PUBLISHING ON STORES. Thanks, Christina
$250 AUD in 7 days
7.0
7.0

Hello!, I am a US-based senior software engineer, and this is exactly the kind of project where reliability matters more than just “making it work” once. For a Coles and Woolworths price scraper, the real challenge is handling page changes, blocked requests, missing fields, and keeping the output clean and usable. My approach would be simple and practical: 1. Confirm the exact data fields, filters, and output format. 2. Build a locally runnable Python scraper with retries, pagination support, and structured CSV/JSON output. 3. Add data cleanup and validation so the results are consistent. 4. If useful, I can also make it easy to extend later for AWS scheduling or logging. I focus on the details because that’s what turns a scraper into a tool you can actually depend on. Relevant examples of work: - Product and price extraction tool for a regional retail site - Grocery data pipeline for a comparison dashboard - Python automation for recurring web collection - AWS-backed scraper monitoring for ecommerce analytics Could you please clarify the following questions to help me better understand the project? 1. Which exact fields do you want collected from each product? 2. Do you need this for specific stores/locations or full category searches? 3. Should the first version stay fully local, or do you want AWS readiness from the start? If you want a clean first version that is maintainable, I can get this moving quickly. James Zappi
$650 AUD in 2 days
6.4
6.4

Hello, I can build a configurable weekly data pipeline using retailer-approved API access or licensed product feeds. The pipeline can normalize product and variant records, associate prices with retailer, location, promotion, and extraction date, remove duplicates, and export structured CSV or JSON. I’d include scheduled runs, retries, logs, and a summary of successful and failed imports. I’d first validate the source fields and store identifiers so location-specific prices are represented consistently. I don’t have a published Coles or Woolworths scraper to show as a direct example. I’d need written permission or an approved data source before collecting their data; for this budget, I’d limit work to a small proof of concept.
$750 AUD in 5 days
6.3
6.3

Build a reliable, locally runnable "Coles & Woolworths Price Data Extraction Tool" that collects weekly Coles and Woolworths product pricing data across Australia. - Build a tool that can be executed automatically every week. - Scrape product data from Coles and Woolworths websites for all available store locations across Australia. - Extract the following fields: product name, package size/weight/volume, current price, unit price (e.g. $/kg), retailer, store location, product URL, promotional price (if applicable), and extraction date. - Capture location-specific pricing, including regional price differences and store-specific promotions, wherever available. - Collect all available products, including different sizes and variants, while avoiding duplicate records. - Export the extracted data in a structured format such as CSV or JSON. - Include logging, error handling, retries, and a summary of successful and failed extractions and uploads. - Make retailer settings, store locations, execution schedule, and AWS configuration configurable without changing the core code.
$700 AUD in 7 days
6.5
6.5

i’ve done very similar recently with Python data pipelines, web extraction, and automated data processing. I’d build this as a configurable weekly job, with clean product records and clear reporting when either retailer changes its site or blocks a request. Do you already have a source for discovering all Coles and Woolworths store locations, or should that be part of the tool? Do you prefer AWS Lambda with EventBridge or a container-based scheduled job? I’d use stable product and store identifiers to prevent duplicates while preserving regional prices and promotions. Retries, structured logs, and per-store status reports will make failures easier to trace. I’ll inspect both retailers’ available data sources, build the extraction and normalization pipeline, then add CSV/JSON exports and configurable scheduling. Finally, I’ll test price variations, duplicate handling, and recovery from failed runs. Best, Dev S.
$500 AUD in 10 days
6.6
6.6

Hello I prefer to write my bids my self not ai Simply I can scrape all these websites easily first using python then converting them into typescript (already took a project like that before) I can reach the apis that these websites get data from and use them directly to be rapid scrapers. We can discuss which websites to start with. Thanks.
$500 AUD in 5 days
6.5
6.5

Canberra, Australia
Payment method verified
Member since Aug 31, 2023
$30-250 AUD
$30-250 AUD
$30-250 AUD
$30-250 AUD
$750-1500 AUD
£20-250 GBP
$250-750 USD
$10-30 CAD
₹1500-12500 INR
$15-25 USD / hour
₹600-1500 INR
₹12500-37500 INR
₹600-1500 INR
$30-250 USD
$30-250 USD
₹1500-12500 INR
$1500-3000 AUD
₹1500-12500 INR
$250-750 AUD
₹12500-37500 INR
$250-750 USD
₹750-1250 INR / hour
$25-50 AUD / hour
$3000-5000 USD
₹12500-37500 INR