Dibatalkan

Write a parser script for a big JSON file

WikiData is project which attempts to collect data about everything in our world. Basically, it will contain information of people, cities, countries, foods, atoms, stars, everything.

You can download their entire database from here in JSON format:

[login to view URL]:Database_download#JSON_dumps_(recommended)

All entries in this database are based on a Q code. The Q code is an unique index number of the specific item. For example, [login to view URL] is the entry for a famous person.

We are interested of WikiData's data on people. For example that of [login to view URL]

Your job is to write a script in any language you want, that will analyze the downloaded WikiData database file(s) and output an SQL insert file.

The script must extract 6 things of entries of people:

1) Person's full name,

2) Person's given name (= first name),

3) Person's family name,

4) Person's gender,

5) Person's country of citizenship,

6) Person's native language.

So, for the same person [login to view URL] These values would be:

1) Full name = Manuel José Bonnet Locarno

2) Given name = Manuel

3) Family name = Bonnet

4) Gender = M

5) Country of citizenship = Colombia

6) Native language = Spanish

And this data would be added to the output SQL file as follows:

INSERT IGNORE INTO wikidata (q_code, full_name, first_name, family_name, gender, country, language) VALUES ("Q5993357", "Manuel José Bonnet Locarno", "Manuel", "Bonnet", "M', "Colombia", "Spanish");

The next found person from the WikiData database file(s) would generate a new line to the output SQL file, and so on, and so on.

Please have the script show some kind of progress indication. For example, the number of rows or entries in the database and the current index or row currently being analyzed, so when running the script, you could see the progress.

The script must ignore all other types of entries than persons. Also, if the person's data is missing any of the 6 data fields (full name, given name, family name, gender, country of citizenship, or native language), skip that person.

The purpose of this task is to be able to run the script, and it will generate a huge SQL file which will insert the person data (name / gender / country / language) to database.

In your bid, please state what language you would use to write this. Scripting language such as Perl, PHP or Python would be preferred.

Kemahiran: JSON, Perl, PHP, Python

Lihat lagi: json file too large, which would be better option to consider in a environment when you have big json file ?, parse large json file python, parse large json file java, parsing large json files javascript, how to handle large json data in android, php parse large json file, split large json file, php script read folder file details, write spider script, perl script unzip zip file, write perl script send logs, text parser script, best script big file, java big xml file parser, write big xml file sax, ajax php script big file uploads, php write output script file, php write json file, node write json file

Tentang Majikan:
( 612 ulasan ) Turku, Thailand

ID Projek: #17802874

41 pekerja bebas membida secara purata $186 untuk pekerjaan ini

rajeshsonisl

Hi there, This is a couple of days worth of work. Can begin immediately. I have a 99% project completion rate and a 4.99 reputation (out of max 5.0 from more than 1140 projects from a period of 12 years. Thank Lagi

$777 USD dalam 10 hari
(916 Ulasan)
8.5
rinsadsl

Hi there I can build this script to import the people data into the database. I can make it using PHP. Looking forward to working with you. Thanks Rinsad

$220 USD dalam 3 hari
(992 Ulasan)
8.7
gangabass

I have worked with huge (> 100Gb) files before that's why I'm sure you'll be impressed with my work. I can provide you Perl or Python script that will parse JSON and generate SQL file.

$140 USD dalam 2 hari
(500 Ulasan)
7.4
expatdev

Hi, I'd like to work with you on this task. I have 15 years of experience with transforming or manipulating very large bodies of data. Also, I am familiar with Wikidata. I can use any of the three languages; among the Lagi

$200 USD dalam sehari
(127 Ulasan)
7.3
zhangyingtai

Hello sir I am a qualified python developer with 8 years of professional experience of web scraping. I can download all json file and make a parser for the json file. I am interested in this project and can start th Lagi

$250 USD dalam 3 hari
(77 Ulasan)
7.1
superman1987417

Greetings. I'm an expert web developer with 8 years of experience. I'm very interested in your project. And I have extensive experience in JSON parsing. I've checked your requirements carefully and feel confident w Lagi

$111 USD dalam 2 hari
(55 Ulasan)
6.8
vano101

Hello I have couple of years of experience with Python and about 1,5 years of experience with PHP I have ideas how to write parser of big JSON file.

$76 USD dalam 3 hari
(271 Ulasan)
6.2
shiningdevelopor

Hello How are you I read your description carefully I have many good experiences in JSON and PHP So I can finish your project quickly and perfectly I am going to develop with PHP If you hire me, I will do my best Lagi

$155 USD dalam 3 hari
(50 Ulasan)
6.1
devi222

Hi Sir, Im a Python Programmer. I have created many Bots to scrape the websites like Alibaba(suppliers data), Linkedin, Facebook, Trulia(real estate, homes), [login to view URL], Amazon(using API), Aliexpress, yellowpages(au, Lagi

$250 USD dalam 3 hari
(43 Ulasan)
5.8
Mickelson

Hi Nice to meet you. I'm python expert, so I will use python to parse your big large json. I have written parse script for such json file (7GB). So no problem and got it you mean. Just let me know if you are inter Lagi

$250 USD dalam 3 hari
(32 Ulasan)
5.5
kaloyan13

I will use PHP with JsonDumpReader library. Can do the job in a day. Thank you......................

$100 USD dalam 2 hari
(56 Ulasan)
5.2
Peterpay

i can do this for you without any problems i'm an expert in data extraction and performance applications.

$250 USD dalam 3 hari
(24 Ulasan)
5.1
shyarakishor1234

Hey My Dear Friend I will build your script in Perl Technology. I have found your requirement regarding build a script which parse json data and build csv file and store that same data in MYSQL Table and i am app Lagi

$222 USD dalam sehari
(24 Ulasan)
4.8
stivens

Hello, I can do this in PHP. Just need to clarify a few things. The script must accept the JSON file(s) then decode the content and output an SQL file with insert statements? Do you have a sample JSON file for referenc Lagi

$180 USD dalam 5 hari
(26 Ulasan)
4.9
wobits

Hello, my name is Wolfgang Backhaus, I am a software developer and system engineer from Germany. I have read Your interesting project offer and want to apply for it. I am a seasoned Perl developer (20+ years) Lagi

$100 USD dalam 3 hari
(7 Ulasan)
4.9
kkc1985612

Hi Dear. I am an expert in parsing of Json, XML. I will provide a great result to you in your deadline. Regards. Mi.

$250 USD dalam 3 hari
(27 Ulasan)
5.0
marenchuk

Now that have read your brief and feel very confident on it. PHP or Python will be used simply to do your job. If you could award me the project, faithful result will be promised. Kind regards

$155 USD dalam 3 hari
(10 Ulasan)
4.4
kchwistek

Hi, I can do it for you. My approach will be: 1) download the WIKIDATA JSON dump. 2) use wikidata-filter to extract only human data 3) write a parser of the provided data in PERL to extract required data and format Lagi

$150 USD dalam 5 hari
(9 Ulasan)
4.3
dslc

I propose to implement a solution in the Perl language. My tentative plan would be as follows: 1) Create a Linux virtual machine (with large disk size - e.g. > 40 GB) for this task + download and extract JSON dump. Lagi

$60 USD dalam 3 hari
(8 Ulasan)
4.3
Priyanshu1

Hi Do you want to download all the data as json and then process it or do you want to download it one by one ?? Do you need the script to run 24X7 ?? I will do it in php using mysql(if you have no preference). A lot Lagi

$250 USD dalam 10 hari
(3 Ulasan)
3.5