1. FastAI was chosen, because of the built-in functionality to handle the entire document via bptt. Doing this in another framework would have required a lot more work, not suitable for a hackathon.
2. FastAI is a wrapper over pytorch – it provides a lot of very sensible defaults and removes a lot of boilerplate. You can use fastai with any torch.nn module – pytorch-bert included. It would be a logical next step. The I did not opt for BERT was that I feared running out of GPU memory.

P.S. Actually, one of the models achieved a score of 0.11 on the dev set, coming in at second place, but it was overwritten somehow by the scorer and we had also lost the submission file so we couldnt resubmit.