<hi xmlns="http://www.tei-c.org/ns/1.0" rend="">For the creation of the Gold Standard of Task 1 (evaluation classification), manual annotation reliability was evaluated on a subset of 250 reviews (~4,000 sentences). Cohen’s </hi> <hi xmlns="http://www.tei-c.org/ns/1.0" rend="italic">Kappa</hi> <hi xmlns="http://www.tei-c.org/ns/1.0" rend=""> </hi> <hi xmlns="http://www.tei-c.org/ns/1.0" rend="">(0.76) indicated a strong agreement between annotators. Overall, 66% of the total sentences were annotated as “evaluation”. Training an SVM classifier on the features generated by the 12 sentiment dictionaries rendered a macro </hi> <hi xmlns="http://www.tei-c.org/ns/1.0" rend="italic">F1</hi> <hi xmlns="http://www.tei-c.org/ns/1.0" rend=""> </hi> <hi xmlns="http://www.tei-c.org/ns/1.0" rend="">score of 0.75 (see Table 2 for details).</hi>