Main Problem

Existing open weight models were not good at greek cultural cues.

Method

They gather training data using GPT5 because of high reasoning capability. THey created a QA dataset with one quetion, 4 possible answers and one final answer. They used LORA training method for training the open weights models due to that being efficient. They experimented with various training parameters and fixed on one due to better training and validation losses.

During benchmark validation across many different greekqa datasets they observed improvements +4-5%, These improvements were only observed in open weight models. Propieraty models were still beating the open weight models.

Final

Their study showed knowledge distillation would help open weights.

Whats not done

Safety alignment, Instruction following remains future work.