<figure xmlns="http://www.tei-c.org/ns/1.0"><graphic url="134_final-1e43d43626c478d3eaea9d691837bbe9.png" /><head>Tabelle 3: Results of the String-Matching baseline using the user-specified knowledge created by the two annotators (D and M) and the difference to the results of the rule-based algorithm using the same knowledge.</head></figure>During the annotation of DROC, our experiments towards inter-annotator agreement revealed, that even human annotators only had an agreement of about 76% B-Cubed (Krug et al., 2018). Achieving a B-Cubed score of about 75% is therefore a milestone where the data seems reliable and we would expect the results to be usable for downstream tasks. An interesting aspect is that there is a large variance of improvement on different texts. Whenever relatively few named-entities that are communicating in a dialog are available in the text, the improvement is high (see <hi xmlns="http://www.tei-c.org/ns/1.0" rend="italic">Amerika </hi>or <hi xmlns="http://www.tei-c.org/ns/1.0" rend="italic">Stechlin</hi>) but the inverse effect occurs, when the author either is very vague with using names and aliases for characters or if there are many characters in the text in general. The quality of the results also depends on the summaries used. Using longer summaries (the ones we used were mostly rather short) or several different summaries per novel will likely lead to better results.