We currently have a video database of roughly 400,000 videos. We need an automatic bot script that will run through every single video, and check it to see if there is really a video there or not. Many of the files will be from youtube or metacafe, so you need to make sure that it does not get banned for accessing the sites to many times. We also have files actually on our server, we should be checking for those as well. I want to be able to run the script from cron, and I want it to produce a summary log that explains what happened in the scanning process.
We have many more projects after this, so the programmer that is working on this needs to be available following this project for more immediately following.