Ramamurthy, RajkumarRajkumarRamamurthyStenzel, RobinRobinStenzelSifa, RafetRafetSifaLadi, AnnaAnnaLadiBauckhage, ChristianChristianBauckhage2022-03-142022-03-142019https://publica.fraunhofer.de/handle/publica/40528510.1007/978-3-030-30493-5_112-s2.0-85072951490This paper explores a simple method for obtaining contextual word representations. Recently, it was shown that random sentence representations obtained from echo state networks (ESNs) were able to achieve near state-of-the-art results in several sequence classification tasks. We explore a similar direction while considering a sequence labeling task specifically named entity recognition (NER). The idea is to simply use reservoir states of an ESN as contextual word embeddings by passing pre-trained word-embeddings as its input. Experimental results show that our approach achieves competitive results in terms of accuracy and faster training times when compared to state-of-the-art methods. In addition, we provide an empirical evaluation of hyper-parameters that influence this performance.en005006629conference paper