Text repair based on n-gram language model

Well, my problem is that I have some texts that have some mistakes or lack some words inside of them. I would like to have a tool that would allow me to repair them automatically using n-gram language model in APRA format. My text in format like one sentence per line in UTF-8, I need the tool to work under Ubuntu linux - I am using version 16.04.

In the text there may be following errors:

- lack of word or words in a sentence (with <unk> symbol): e.g. "Ala ma <unk> i dwa psy." should be corrected to "Ala ma kota i dwa psy.". NOTE. There can be many <unk> in one sentence.

- wrong for form or inflection: e.g. "Ala kupił zielony samochód." and should be "Ala kupiła zielony samochód."

- (very rare) totally wrong word in text: e.g. "Rąb ma pole o wymiarach." and should be "Romb ma pole o wymiarach."

All this should be repaired based on n-gram language model (so statistical model). Such a model has for each n-gram probabilities such probabilities usually are calculated as conditional ones as described here [url removed, login to view] and here [url removed, login to view] . So in fact the goal is to find the most probable sentence with given words and find missing content and input it.

I will use 3-gram, 4-gram, 5-gram, 6-gram and 7-gram models. So for all of such models the tools should be able to provide results based on all n-grams possible.

Here is sample n-gram model you can use it for testing: [url removed, login to view]!th83lRII!ZS4Mr3NOxrMst941yQXpDuobHcA6yUgJJKMNu9DUBJE

This is very small LM so the tools should be fast, work in parallel for files like hundreds of GB.

Kemahiran: Pengaturcaraan C, Java, Perl, Pengaturcaraan , Python

Lihat lagi: i need a cad drawing from sketch, i need someone that is a sketch cartoon artist to do cartoon sketches for 56 total scenes cartoons, i need freelance sketch up work in dubai, i need a sketch artist and logo designer, i need a sketch artist, i need an artist to sketch a profile picture, i m certified in technical support i need found a job, i can sketch but need a designer, i m trying to find a website that will help me create my own website i m not technical so i need an easy to use one any suggesti, i need someone who can sketch my fashion designs, i need someone to design/sketch, i need a sketch artist mangarf kuwait, need sketch tree, text basedbrowser based rpg, fashion sketch needed, merge text files based common field, php text turn based game maker, mobile repair based websites, cms text files based, text client based mmorpg

Tentang Majikan:
( 146 ulasan ) Czestochowa, Poland

ID Projek: #15630255

2 pekerja bebas membida secara purata $250 untuk pekerjaan ini


I am very proficient in c and c++. I have 16 years c++ developing experience now, and have worked for more than 6 years. My work is online game developing, and mainly focus on client side, using c++ under windows envir Lagi

$250 USD dalam 7 hari
(114 Ulasan)

-------------------------------------------------------------NLP Expert!---------------------------------------------------------------------

$250 USD dalam 3 hari
(7 Ulasan)