Smart card-based remote authentication schemes are widely used in multi-medical-server-based telecare medicine information systems (TMIS). Biometric is one of the most trustworthy authenticators, and is presently being advocated to use in the remote authentication of TMIS. However, most of the existing TMISs consider a single-server-environment-based authentication system. Therefore, patients need to register and log into every server separately for different services. Furthermore, these schemes do not employ error correction technique to remove the errors from biometric data. Also, biometrics are inherent and demand diversification to generate a revocable template from inherent biometric data. In this paper, we propose a mutual authentication and key agreement scheme for a multi-medical server environment to overcome the limitations of the existing schemes. In the proposed scheme, a cancelable transformation of the raw biometric data is used to provide the privacy and the diversification of biometric data. The errors of the biometric data are corrected with error-correction techniques under the fuzzy commitment mechanism. Formal security analysis using the widely accepted Real-Or-Random (ROR) model, the Burrows-Abadi- Needham (BAN) logic and the Automated Validation of Internet Security Protocols and Applications (AVISPA) tool concludes that the proposed scheme is safe against known attacks. We also compare the computation and communication costs of our scheme to evaluate the performance with the others.