The EMH backup module was a 24th century computer memory storage device carried on Starfleet vessels equipped with the Emergency Medical Hologram program. It was capable of storing data too complex or too large for contemporary systems. As such, it was used to store backup versions of the EMH, on a frequent basis, in case it was lost or damaged.
The USS Voyager's backup module was stolen during an attack and recovered centuries later by the Kyrians. This version of The Doctor helped to reshape Kyrian and Vaskan history and served as the Kyrian / Vaskan surgical chancellor for many years before finally taking a small vessel and setting course for the Alpha Quadrant in an attempt to trace the path of Voyager. He claimed to have "a longing for home." (VOY: "Living Witness")
In the String Theory trilogy of novels, it was revealed that there were several backup EMHs, most of them based on different versions of The Doctor according to when the information was "saved." In the second novel in the trilogy, Fusion, a Nacene who had infiltrated the crew by altering their memories- creating the illusion that she was Janeway's sister, Phoebe Janeway, who had accompanied Voyager on their original mission to create a portrait of the Badlands and remained with the crew ever since-, was forced to shut down The Doctor and activate the first backup version- essentially the "original" EMH activated in "Caretaker"- as The Doctor knew that she was not who she seemed (Significantly, 'Phoebe' had initially contemplated simply altering his programmed memories, but her analysis of The Doctor showed some undefined quality within him that made him more than a simple computer program, suggesting that The Doctor's development at this point in his existence had advanced to a point where he had acquired some kind of 'soul'). Despite his lack of practical experience, the new EMH was still able to confirm that the crew's memories had been altered, as well as explain why Harry Kim and Naomi Wildman had not been affected by the alteration, as the two were slightly 'out of phase' with the rest of the crew as a result of them being duplicates created during the events of VOY: "Deadlock", before B'Elanna Torres was able to restore the original Doctor.