I currently manage a large news and entertainment site on Joomla, over time its becoming more and more work for a single person to update due to mass amount of gossip, news and music videos being released.
I have decided its best to automate a large section of the site, since most news and gossip is usually from the same news providers. The only difference being that title and first paragraph usually change. I have seen a number of components on Joomla and Wordpress which have the possibility to scrape sites either via rss or url and obtain the full page's output. The problem with most of them is they dont all work well, scrape well or dont reword the content. I am looking to move the site to Wordpress or keep it on Joomla but that will be a separate project. If we can get the functionality right the migration wont be too much of an issue.
I see two options as to how we achieve what I would like;
1) You use a existing extension on WP/Joomla and customise it to achieve the desirable listed below
2) You build a independent php based site that scrapes and directly inserts into the database for Joomla / WP the content
- Scrape Article Title, Text and Post Image
- Auto Rewrite Title, Text using Google or other free service (option to disable)
- Option to include source url at bottom of article
- Scrape on Schedule and on Demand
- When scraping on demand option to select which articles to scrape from results
- Limit number of items
- Scrape multiple sites and file in respective categories
- When scraping set author on article to predefined for scrape, ie Bollywood News Scrapes would have a set name and Technology News would use a different author
- Remove any link backs to original site and any link backs mid article
- Copy Meta Tags etc and modify, Auto Tag new Content
- Videos Scraping (optional) - Scrape a specific websites music videos section, scrape the music videos title, youtube link and insert into seperate database and automate posting
I expect some level of standards from the coder, therefore Honesty is very important, if you are modifying a existing component or know one that does the job and you need to train the template then you must advise this in sources and also document the modifications otherwise I expect the code to be fully commented. Simply if i ever need to update the code or a bug exists another developer should be able to know what was done.
Payment will be made in full on completion of the project, simply we test on a server i can provide or you provide, it does what I want, payment is made in full and you provide the full source code and files.
15 pekerja bebas membida secara purata £211 untuk pekerjaan ini
Hi sir, I am scraping expert, I have did too many similar projects, please check my feedback then you will know. Can you tell me more details? then I will provide demo data for you. Thanks, Kimi
Hi, My name is Huy. I have worked with Joomla for 4 years. As you can see in my profile, I have completed many Joomla projects. Let me help you a hand.