Twitter Archive Site using API

The goal is to take daily information from Twitter via the API (not screen-scraping), put it in a database, and display it on a basic webpage. This PHP/MySQL or RoR/MySQL app should: 1. Maintain a list of select twitter usernames/IDs for daily archiving. 2. Pull profile information, including friends / followers count, url, bio, image, etc. 3. Pull tweets from user's timeline. 4. Store in DB 5. Display in a simple webpage. 6. Repeat archiving on a daily basis working within rate limits. Update: March 25@12:35p EDT: Changed bid deadline to April 1 (moved up from April 6) Update: April 1@9:52p EDT: Changed bid deadline to April 5 to conclude discussions with current bidders. ## Deliverables What we need done: Create a PHP/MySQL or RoR/MySQL app that: 1. Allows us to add users to the database so they can be archived. Users should be added using their Twitter usernames via a form. These should store separately in the database so we can remove users we want indexed via the db. 2. Pulls user profile data: Based on the list of usernames in the db, pull user data from Twitter. This data should include the information found in their profile (API link: <[login to view URL])> and a datetime stamp with when the data was pulled. Each day a new entry should be added for a user (basically, we won't delete/replace old data - we'll just add new data to the end of the table). 3. Pulls user tweets: Based on the list of usernames in the db, pull user tweets and store them in the db. This data should include the information available via the API (we believe this call is the best, but open to suggestions: <[login to view URL]>). For this part, the data pull should include as far back as possible without repeating any tweets. So, if a new user is added, the app should attempt to pull as many of their tweets as possible. If a person has been on the list for a while, it will only pull the tweets since the last time they were pulled. (so, in the API you might use the since_id parameter to limit the tweets being pulled). 4. Display information: There are two main pages for the app: a) homepage and b) a user page. Just pull the data and show it - nothing fancy. Design will come later in time. 5. Cron job: This is probably the one part we are open to the most feedback on. We want this app to pull the data for us with a cron job or something. We'd like something to do steps 2 & 3 above on a regular basis (daily for now). 6. Update now button: On the user profile page, I put an "update now" button. The idea with this is it would force the data pull to happen for this user page. 7. Relax and enjoy the app. DB Table stuff: So, based on the above, we're anticipating the following tables (if you have suggestions on how to optimize this or change it, we're open to listening...as long as the data is all still there): 1. User list table (based on #1 above - a list of all users being indexed). 2. User profile table (based on #2 above - user profile data being archived daily with new entries and a date/time stamp). 3. User tweets table (based one #3 above - user tweets being archived). Design stuff: Not looking for a fancy design at all - just really something that shows the data. We'll layer a design later in time. Rate limit stuff: Twitter's API has rate limits - currently about 350 / hour for the REST API. They offer a Streaming API - though not sure this would be appropriate for the app. We'd like the developer to think about this and propose a solution. This may mean a queuing system for handling requests to ensure they all get met; not sure - let's hear your thoughts. Other stuff: At no time should a person need to login for this app. There's no authentication for users, accessing Twitter accounts, admins, etc. So, don't worry about it. Please ask any questions you have prior to submitting your quote. We are looking to work with someone on this project and potential expansions moving forward. We will do our best to respond to all questions promptly and clarify the project on the posting as needed.