Hi, the project consists of 2 elements: web backend and mobile app
There are a lot of news sites that provide NO RSS feed and can't be used with regular rss feed reader.
Visiting all those sites takes a lot of time. There is no automatic feed update. No easy way to aggregate all news sources.
Build a web backend that scrapes a list of news sites using individually customized web scraping script for each site. Create a mobile news reader app for android and ios to easily fetch those news.
1. Web scraping backend
- I want to have a web based backend system that scrapes a list of web sites every 10 minutes for new articles, the article main text, the article cover image, article date, article category, number of views.
- The web scraping script is customized for each news site, since they often use different formatting and cms. For this project I want to start with only 2 different websites for which the scraping works. I have a team of web scrapers, who can add more later, but the overall backend system is scope of this project and needs to be created.
- An article scraped is sorted by category, date, and views and is stored in a mysql database on the backend server.
- The web scraping script runs every 10 minutes to check for new articles, if an article is already present in the database, it is not ignored and not scraped. To check if an article is new or not it compares article name and date and source url.
- Since the article text and article cover image is stored on the backend server, it needs to be send to the mobile app either by push, which invokes a push notification in the icon, or by starting the app, which then downloads the article context in json format.
- There needs to be an efficient way to cache the results of the articles, either as an static file or as a cache file. The mobile app should not cause a backend database query everytime it loads.
The backend database is NOT queried, everytime the mobile app asks for updates.
- 1 medium server with the backend scripts should be able to serve up to 10.000 mobile clients pro minute, so careful design and delivery of the articles to the mobile phones have to be considered.
2. Mobile news reading app
- The mobile app should be built on a platform that allows rapid deployment on iOS as well as Android.
- The mobile app's design should be very similar to Feedly news reading app
Please download the feedly app to understand the scope of the mobile development part:
[login to view URL]
[login to view URL]
- The mobile app will allow a user to select a predefined list of source sites to follow and read the articles.
- The display of articles should be very similar to Feedly. Clean and minimalistic, yet structured and clear.
- The mobile app allows a user to Save, and Like, and Share an article.
- A top list of news articles with the most views will be shown in one tab. One can influence the global order of those articles by Liking or Disliking the article.
- Articles will be shown in full, but the source site url can be opened by scrolling to the bottom of the article.
- At the bottom of every page will be advertisement banner, maybe admob, so banner integration is important.
- Optional, if possible, at the bottom of every article will be a list of comments that other users of the app can post.
- Debugging and testing before delivery are required! Full source code for all scripts and apps are required and included in the price.
- Please install the Feedly app to understand about the scope of the project. The look and feel should be very much like that. Performance of the app should be fast, lightweight, in order to run on older, slower phones (Android 4.4+).