Railway systems have evolved considerably in the last years with the adoption of new communication technologies. Aiming to achieve a single European railway network, the European Rail Traffic Management System (ERTMS) emerged in Europe to substitute multiple and noninteroperable national railway communication systems. This system and its security strategies were designed in late 1990s. Recent works have identified vulnerabilities related to integrity, authenticity, availability, and confidentiality. In the context of defining effective countermeasures to mitigate potential vulnerabilities, these vulnerabilities have to be analysed. In this article we introduce a framework that attempts to challenge ERTMS security by evaluating the exploitability of these vulnerabilities.