I updated the paper with conf matrix without normalization. The result on test set is different because I pretrained the model because of issue with the keras tokenizer.

I will try to explain what I meant by “The first one was using function from R*R -> R that holds h(a,b) != h(b,a) and add this as feature”. The idea was to create a function that converts an ordered pair of labels to just one component. and use it as a feature, because some classifiers we tried create more features based on combinations of the initial and thus I decided that they will only create noise.

Yes , Preslav you are right our confusion matrix shows normalized score between 0 and 1, however it can be easily changed by param inside the notebooks we shared.