Modifying files of BigCode Evaluation Harness

Ditutup Disiarkan 3 bulan lepas Dibayar semasa penghantaran
Ditutup Dibayar semasa penghantaran

Currently, while generate and evaluate on a task using bigcode evaluation harness, it generates a .json file something attached below. The task is to try appending a single hard-coded prompt to prompts in TokenizedDataset in lm_eval/

First, understand what the exact structure of the resulting prompt will be like.

E.g. “Prompt: print hello world \n\n Answer: print(‘hello world’) \n\n Prompt: read a file \n\n Answer: “; vs “Intent: print hello world \n\n Code: print(‘hello world’) \n\n Intent: read a file \n\n Code: “.

a. “# prints hello world”; what separator to use between the prompt and the answer

b. check the docs on huggingface for models - to see if they have examples of ICL (in-context learning)

c. check the papers of the models (codegen) - for any examples how to use ICL with the model

Python Bahasa Semulajadi GPT-3 Deep Learning NLP

ID Projek: #36717006

Tentang projek

2 cadangan Projek jarak jauh Aktif 2 bulan lepas

2 pekerja bebas membida secara purata ₹1300 untuk pekerjaan ini


Objective Highly energetic, motivated and enthusiastic teenager able to quickly learn and handle varied responsibilities is seeking employment opportunities to further enhance learning abilities and to gain experience Lagi

₹1300 INR dalam 7 hari
(0 Ulasan)