Philip Hensher on how an impassioned, chaotic group of amateur 19th-century composers created the first distinctively Russian music