El Fardou Ben Nabouhane put the islanders ahead early on, and Ghana then went down to 10 men when Andre Ayew was shown a straight red card.
Ahmed Mogni doubled the lead before the Black Stars fought back to make it 2-2.
However, Mogni tapped in his second in the 85th minute to inflict Ghana’s first group-stage exit since 2006.
The result in Group C will go down as one of the biggest shocks in Nations Cup history as Comoros scored at the finals for the first time and then went on to record their maiden victory.
Incredibly, the side ranked 132nd in the world now have a chance to reach the knockout stages as one of four best-ranked third-placed sides even though the Coelacanths lost their first two games in Cameroon.
The tiny island nation with just under a million inhabitants – and more famous for its history of political coups than its football – had only won its first Nations Cup or World Cup qualifier in 2016 at the 20th attempt.
A squad bolstered by members of the diaspora, with many plying their trade in the lower leagues in France, lost just once to achieve a maiden qualification and Comoros have now beaten one of the continent’s giants.
Ghana had put in disjointed displays in their first two games, a 1-0 defeat by Morocco and a niggly 1-1 draw with Gabon, and finish bottom of the group table despite putting in a spirited display following Ayew’s 25th-minute sending off.
Their fans in the Roumde Adjia Stadium in Garoua looked stunned at the full-time whistle, being left to contemplate a rollercoaster second half which ended their bid for a first Nations Cup title since 1982.
Their exit will increase the pressure on coach Milovan Rajevac, who returned for a second spell in charge in September.
The Serb led the Black Stars to the Nations Cup final in 2010 and the World Cup quarter-finals the same year, and guided the West Africans through to Africa’s play-offs for this year’s World Cup.
However, Ghana only won their qualifying group ahead of South Africa on goals scored and are unseeded for the play-off draw – where they will face one of Senegal, Morocco, Tunisia, Algeria and Nigeria for a place in Qatar.
Comoros, meanwhile, celebrated their three points heartily and now await the final round of matches in the remaining three groups to see if their stunning victory can extend their stay at the tournament.