Phenetic and ecological plasticity in Arvicola has caused a long-standing dispute over the number of species within the genus, which is currently thought to consist of two aquatic (sapidus, amphibius) and one fossorial species (scherman). We used mitochondrial cytochrome b (cytb) gene sequences to reconstruct phylogenetic relationships between the fossorial and the aquatic water voles from the various regions of their European and Asiatic range. These two types differed morphologically and exhibited allopatric ranges. Our study provided 50 new haplotypes, generating a total dataset of 70 different water vole cytb haplotypes. Phylogenetic reconstructions retrieved two major lineages that were in a sister position to A. sapidus: a fossorial Swiss lineage and a widespread cluster, which contained aquatic and fossorial water voles from Europe and western Siberia. The phylogeographic architecture in water voles is explained by Quaternary climatic dynamics. Our results show that A. scherman in its present scope is not a monophyletic taxon.