Countering German Hate Speech: Co-designing and Evaluating an Expert-Grounded RAG Tool

Social media moderators have to face high volumes of German hate speech, yet drafting counter speech (CS) that contest harmful claims is time-consuming and emotionally burdening. This paper presents the co-design and evaluation of a browser-based CS assistance tool developed together with experts from Civic.net. The system implements a custom retrieval augmented generation (RAG) pipeline that grounds outputs in Civic.Net’s internal strategy guides. Out of these guidelines 15 hate-speech categories are encoded as structured knowledge chunks and retrieved to steer generation. The interface provides four practitioner-defined styles, length controls, and a generated problem statement describing what is harmful in the input comment. We report a formative usability test and two expert evaluations comparing gemma2-9b-it, llama3-70b-8192, and ChatGPT 4o. LLaMA 3 performed best for formal CS generation, but only 22% of its outputs were usable without edits; across models, most outputs required modification before being usable. In style-focused ratings, humorous variants received the highest average scores and model preferences varied between the evaluators. Following implications for tools supporting moderation were discovered: offer multiple tones by default, expose retrieved strategy snippets for verification, and support lightweight post-editing for adjustable usage.