Rapid advances in technology have made it difficult to distinguish fake voices from natural human voices, bringing serious security risks such as fraud and manipulation. While most studies in the current literature focus solely on binary classifications that question the authenticity of the voice, tracking studies that identify the source of the fake voice and issues of data imbalance are often overlooked. Since accurate source attribution is critical for forensic investigation and preventing misuse, this study proposes a two-stage hierarchical detection system to address both detection and attribution challenges. The system is evaluated on a comprehensive dataset containing real voices and fake voices generated by eight different models, including FlashSpeech, VALLE, and OpenAI. To tackle class imbalance and improve generalization, the study utilizes a 2189-dimensional hybrid feature vector derived from models like YAMNet, Wav2Vec 2.0, and Resemblyzer, combined with 17 different data augmentation techniques that enhance the model's representational power. In the comparative experiments involving nine machine learning algorithms, the SVM (RBF Kernel) model achieved the highest accuracy of 97.97% in the first stage, while Logistic Regression demonstrated the best performance in the second stage with 93.98% accuracy. Considering the averaged accuracy across both stages, SVM (RBF Kernel) exhibited the strongest overall performance with 95.51%. These results demonstrate that the proposed hierarchical approach, coupled with hybrid feature extraction and data augmentation, offers an effective solution for cybersecurity and digital forensics, particularly in preventing the misuse of synthetic voice generation and in digital content verification processes.