This review is devoted to spin glasses, i.e., disordered magnets with randomly distributed ferro- and antiferromagnetic exchange interactions. Extensive experimental data and the theory developed for a simple model with an infinite exchange interaction range show that, as the temperature is reduced, the competition between different types of disorder leads to a nonergodic (or quasi-nonergodic) state in spin glasses. The fundamentals of the statistical mechanics of nonergodic systems are established. The review also examines the connection between the problem of spin glasses and those of combinatorial optimization, as well as the use of spin-glass models in biology. Attempts to construct a theory of real spin glasses with a distance-dependent exchange interactions are discussed.