PARIS — There are few sights more iconic than the Eiffel Tower and the Olympic rings. But not everyone agrees they belong together — least of all the family of the man who designed the landmark, which is now at the center of a high-profile fight over its future.

The multicolored logo adorned the Eiffel Tower during the Paris 2024 Games, just as they had hung from London’s Tower Bridge in 2012 and the Sydney Harbour Bridge in 2000.

Paris Mayor Anne Hidalgo revealed last week that she plans to keep the rings there indefinitely, reflecting the extent to which this famously cantankerous city was swept up with Olympic bonhomie.

The proposal sparked outrage, with political opponents accusing Hidalgo of trying to cash in on the event’s success ahead of a re-election bid and heritage campaigners saying she risked besmirching a piece of the city’s architectural history.

Now, those voices have been joined by the family of Gustave Eiffel — the engineer who designed the eponymous tower for the city's Universal Exposition of 1889.

“While the rings were celebrated on the Eiffel Tower during the Games because the iconic monument was a central figure in the event, the rings were never part of its original design,” Myriam Larnaudie-Eiffel, the engineer’s great-great granddaughter, told NBC News in a phone interview. “The family will only allow the rings to remain until the end of this Olympic year.”

It has become customary for host cities to place the rings in iconic locations, such as the London and Sydney bridges, as well as Odaiba Beach for Tokyo 2021 and Madureira Park for the Rio 2016 Games.

But they are almost always taken down afterward, as was always the plan in Paris.

However, Hidalgo announced in the French media Aug. 31 that the rings would stay on the Eiffel Tower, until at least 2028 and perhaps beyond. She said that she had the backing of the International Olympic Committee — and that the decision was hers to make.

After days of mounting criticism, the Eiffel family weighed in.

The Association of the Descendants of Gustave Eiffel argued in a lengthy statement Sunday that the move would both ruin the simplicity of the structure’s design and impinge on its neutrality.

The 95-foot-wide, 43-foot-high rings create a “strong imbalance” and “substantially modify the very pure forms of the monument as designed by Gustave Eiffel and his teams more than 135 years ago,” the statement said.

Crucially, it said, the rings hang between the second and third floors of the iron latticework, breaking its clean lines and obscuring the negative space through which the rest of Paris can be seen.

It's not just design worries that are prompting the family's ire.

They also argue that the rings’ presence “contradicts the neutrality and the meaning acquired over the years by the Eiffel Tower, which has become the symbol of the city of Paris and even of the whole of France throughout the world.”

They have engaged Dauzier & Chappuis, a French law firm specializing in intellectual property.

Hidalgo's office said in a statement that the mayor was in discussions with the IOC and national government about how to press ahead with her decision, but declined to comment further.

Asked about proposal and the Eiffel family's criticism, the IOC said in an email that it "warmly welcomes" the idea to "make the legacy of the sensational Olympic Games Paris 2024 accessible to as many people as possible." It said it was conducting a "feasibility study," the results of which would be available "in the next months."

The Olympic Games are and always have been deeply political events: used to boost the ambitions of host politicians while promoting the branding of the Games’ many global sponsors.

Hidalgo, the mayor, argues that keeping the rings would reflect the emotional and perhaps even spiritual change many Parisians felt during the Olympic and Paralympic summer.

Many locals fled town, fearing tourist chaos, high security and transportation logjams. However, those who stayed said they had never known the French capital overwhelmed with such a feeling of lightness and joy. Many backtracked on the exodus and returned to join the party.

“The French have fallen in love with Paris again,” Hidalgo told the newspaper Ouest-France. “I want this festive spirit to remain.”

In truth, the tower is a building that has always been mired in controversy and subject to unpopular changes.

Supposed to be a temporary installation for an 1889 exhibition, it was deeply criticized and faced protests against its construction.

It was saved only because Eiffel made the case that it could be used for scientific experiments, mounting barometers and lightning conductors at its summit. It later gained utility as a radio mast.

The tower has also not been without its tweaks over the years, most notably in the 1920s, when it became a giant billboard for the French car company Citroën.

Nevertheless, the Eiffel family say they won’t climb down without a fight.

Nancy Ing reported from Paris and Alexander Smith from London.

This article was originally published on NBCNews.com