The chain of command was a ranking system used by militaries and other organized groups. It represented the line of authority down which orders are passed from one officer to another, and also the line down which passed command of a particular vessel or installation.
Aboard a starship, or most starbases the chain of command began with the assigned commanding officer, usually an officer of the rank of captain, or in some cases commander. If the commanding officer, for any reason, was unavailable to command his or her post then those responsibilities passed down the "chain" to the first officer, then the second officer, etc.
Between the launch of the Enterprise NX-01 and the late 23rd century, the position of first officer, or executive officer (abbreviated as XO) was filled by the highest ranking officer assigned to the ship aside from the captain. The position was not a separate assignment as it would later become. The designated first officer would perform their normal duties until it became necessary for them to take command.
This had changed by the late 24th century, when the XO's position had become a completely separate position in a starship's crew held by an officer of commander or lieutenant commander rank grade who held no other position aboard the vessel.
After the executive officer, command responsibilities passed down the chain from officer to officer as necessary in accordance with each officer's rank. And it was common practice for a commanding officer to designate a second officer to take command if both the captain and first officer were unavailable or unable to perform their duties.
This aspect of the command chain functioned similarly aboard a starbase, or space station, as well as in a planetary installation.
More broadly, the chain of command could refer to Starfleet, in general, with the chain beginning with the admiralty. And the Federation Council, and/or the President of the Federation could be considered to be at the ultimate top of the chain.
When Spock inquired to Dr. McCoy if he had noticed Captain Kirk acting strangely lately, McCoy replied, "the man on top walks a lonely street. The chain of command is often a noose." In response to McCoy's words, Spock told him to "spare me your philosophical metaphors." Spock then pointed out Kirk's transfer of Kevin Riley to engineering, and McCoy explained that "he's the Captain, he can transfer whoever he pleases," adding "you can look that up in a hundred volumes of space regulations somewhere." (TOS: "The Conscience of the King")
Benjamin Sisko learned many things from Captain Leyton aboard the USS Okinawa, including that "a good officer has to respect the chain of command." Leyton believed that without it "Starfleet would cease to function and we wouldn't stand a chance against our enemies." (DS9: "Paradise Lost")