Chat is used in many learning scenarios and platforms for synchronous communication support. Up to now evaluation of chat communication requires time consuming manual coding and analysis of the chat transcript. In this paper we present a method that combines manual and automatic steps: (1) chat transcripts are manually referenced, i.e. for each chat contribution it is determined to which previous contribution it refers to. (2) the referenced chat transcripts are structurally analysed by calculating different measures. The results of this structure analysis help to evaluate the learning chat and indicate where further (manual) analyses might be helpful. In addition, the ability of chat to support a certain type of learning scenario can be evaluated. We then discuss how chat can be improved by providing functionality for explicit referencing to the participants during the chat. The evaluation of a university seminar in the winter term 2004/2005 that used the tool KOLUMBUS Chat shows that references are used to different extents and not continuously. We analyse the reasons for (not) using explicit references. The results provide hints about the technical and organisational design for learning chats using references.