we have webstores that use mysql database to hold products info. we are purchasing product data and content which is delivered to us in a txt format. for each product we get two files, one has data on product specs and pricing ,etc in about 40 fields, and the other with links to images of products that reside in the servers of the company that provides us the data ( about 8 fields) . both txt files have a common item number that makes it easy to match both files per each product.
the webstore has a utility to upload via a csv file, but we have about 500,000 products, so its a problem changing from txt to csv and then upload to the mysql using the cart utility
therefore we wish to populate the mysql database and its tables for the webcart directly from the txt files
project is divided in two stages is for creating a software utility that will :
first stage, initial load of data to webcart (about 500,000 items)
[url removed, login to view] both txt databases
[url removed, login to view] about 25 fields of txt databases 25 in the first one and 5 in the second one
3 Map about 30 fields total in about 5 mysql tables of the webcart database
4 Load data from the 30 txt field to the 30 mysql field.
fields will remain allways the same so hardcode is ok
second stage, monthly product maintainance
after initial load of all products, we get every month a update file also in txt, of changes in prices, additions of new products and deletions of discountinued products.
provider of data has specific fields to manage which products will have changes.
the second project will be then to make a software utility that:
1 change pricing in database
[url removed, login to view] discountinued items in database
3. create new products in database.
fields will allways remain same, so hardcode is ok
we have both txt files and webstore in the same linux dedicated server, so software will be also there.