Activist investor Legion Partners Asset Management has called on Guess to remove founding brothers Paul and Maurice Marciano from the board, saying sexual-misconduct allegations against one of them are holding the iconic fashion brand back.

Legion has said that Paul Marciano’s legal troubles in recent years, namely a host of complaints from models alleging sexual misconduct, are undermining the company’s turnaround efforts, according to a letter seen by the Wall Street Journal. The activist added that Maurice Marciano should also be ousted for not sanctioning his brother’s behavior.

Paul Marciano has been publicly accused of sexual misconduct by at least 11 women, including four in the last year, Legion noted. In 2018, Paul Marciano stepped down as executive chairman after being accused of harassment by model and actress Kate Upton.

In 2019, when Guess hired Carlos Alberini as CEO, Guess said Paul Marciano would remain chief creative officer at the request of the board. His brother Maurice was acting as chairman at the time.

Guess confirmed the Legion missive but said the allegations brought up by the activist were largely investigated in 2018, suggesting the matter was put to rest with Paul Marciano’s step back from the executive chair position. The company also pointed out several steps to boost shareholder value in recent years, including the $327 million spent on buybacks from 2019 through October 30 last year.

In its letter, Legion, a 2.5% shareholder in Guess, hailed the retailer’s management for cutting costs and improving the supply chain but said the company is unlikely to reach its potential with the “legal, reputational and moral risk” posed by the Marciano brothers’ continued involvement.

Legion pointed out that Guess and Paul Marciano were recently sued by their underwriter Beazley Insurance Company, which seeks to absolve itself of any responsibility for covering claims related to Paul Marciano’s alleged “pattern” of “wrongful acts.”

The brothers founded Guess more than four decades ago and control about 38% of the stock. The company’s shares rose 7.7% Tuesday but are still down 6% over the past year.

Legion, which manages about $455 million in assets and returned 31% last year, is also engaged in a campaign against the proposed sale of SurveyMonkey parent Momentive Global to Zendesk, a software services company.