I need a script to process very large text files - 100,000,000 lines per text file. Sometimes the data gets corrupted with odd characters, long strings of numbers, foreign characters such as Japanese and Chinese kanji etc. So that text "cleaning" is one function but I also want to be able to do, at the same time, a find and replace of several characters/words/numbers of which I have a list. I will need to be able to add other characters - I have a list, but instead of individual characters maybe it is best to simply eliminate all characters that are not in the English alphabet or numbers 0-9?) and probably one or two other functions to sort and remove duplicates. I know there are lots of text processing apps that offer some but not all of these functions but I have to do it all by hand and they can't handle really large text files or even an automated process that selects the next file in the que would be good. I think this is an easy app to write for the right person.
19 pekerja bebas membida secara purata $186 untuk pekerjaan ini
Hello, i can do small desktop application the loads data from your input files then do the required processing remove unwanted characters do replacing i will deliver source code contact me on chat Thank you
Hello, I will use linux command line and tools and Python for this project. I can clean up the characters and add replacing functionality. I am the master of automation.