A protectorate was a world or region which lived under the protection (or control, if the term was used euphemistically) of another more powerful group.
The Evora homeworld was a protectorate of the United Federation of Planets. (Star Trek: Insurrection)
According to a Deleted scene, Ba'ku was also slated to become a protectorate.
Raatooras, an Arin'Sen colony, was a protectorate of the Klingon Empire before the Klingons abandoned the planet. (ENT: "Judgment")
On Kelemane's planet, the Central Protectorate was located at the southern hemisphere. (VOY: "Blink of an Eye")
Haven, seemingly not a Federation member world, had a treaty with the Federation which required the Federation to protect it, and thus may also have had protectorate status.