I need a web app/script to process very large text files - 100,000,000 lines per text file. Sometimes the data gets corrupted with odd characters, long strings of numbers, foreign characters such as Japanese and Chinese kanji etc. Text "cleaning" is one function but I also want to be able to do, at the same time, a find and replace of several characters/words/numbers of which I have a list. (see attached) I will need to be able to add other characters - I have a list, but instead of individual characters maybe it is best to simply eliminate all characters that are not in the English alphabet or numbers 0-9?) and probably one or two other functions to sort and remove duplicates. I know there are lots of text processing apps that offer some but not all of these functions but I have to do it all by hand and they can't handle really large text files or even an automated process that selects the next file in the que would be good. I think this is an easy app to write for the right person.
The following must be able to be toggled ON/OFF
So the requirements are:
1- Clean up
2- Sort lines
3- find and replace
4- remove duplicate lines
5- remove spaces in front and at the end of each line
6- negative words (delete lines containing these words)
7- positive words (Keep only lines containing these words )
8. Automatically load and process additional text files as needed.
I'll complete your project satisfactorily your requirements. You will get quality work at an affordable price. I can start work right away. I will be looking forward to hearing from you. Thank you.
16 pekerja bebas membida secara purata $166 untuk pekerjaan ini
Hello. I am very well experienced in big data analysis job with PHP & PYTHON. I am sure I will fulfill your requirement perfectly. Please ping me to discuss more. I am ready to start right now. Regards.