There is “no question of going back” on the Brexit withdrawal agreement, French Foreign Minister Jean-Yves Le Drian said on Monday.