The soldiers staged a mutiny against the government's handling of a rebellion in the country's north, which has displaced thousands of people.
Dozens of troops stormed the offices of the state radio and television broadcaster in the capital Bamako, firing off rounds inside the building.
Programs went off air after around 10 minutes of automatic gunfire, but returned within a few hours.
Crowds of youths, some cheering and burning tyres, gathered nearby.
"We now know it is a coup d'etat that they are attempting," a defence ministry official said, asking not to be named.
Calls for calm poured in from abroad as the United Nations, France and United States expressed alarm at the events unfolding.
Sustained gunfire rang out at the palace late on Wednesday (local time) and one of the mutineers told AFP the situation was "largely to our advantage on the ground at the palace".
Elite paratroopers known as the Red Berets had taken up positions around government buildings late in the afternoon, but were called back to defend the presidency, where an advisor to president Amadou Toumani Toure said earlier that the head of state was holed up.
Meanwhile, an independent source said President Amadou Toumani Toure, earlier holed up in the palace as shots were traded outside between the renegade soldiers and loyal guards, had managed to leave the premises.
On Wednesday, an adviser to Toure told AFP that the president was holed up inside the palace, under protection of his elite paratroopers unit known as "Red Berets," who took up position outside government buildings.
Also on Wednesday, renegade soldiers seized the state broadcasting station. Anger had erupted among the soldiers who say they are fed up with a lack of resources and proper weapons as they battle Tuareg insurgents waging a two-month-old war in the north of the country.
The Tuareg rebellion is part of a decades-old bid for independence, and this offensive, the first since 2009, was boosted by the recent return of heavily armed men who fought in Libya for slain dictator Muammar Gaddafi.
France and the US expressed alarm at the unfolding events and urged a peaceful resolution.
"The situation is currently unclear and unfolding quickly," State Department spokeswoman Victoria Nuland said, urging US citizens to stay indoors.