It was around 475 that the bishop moved from Alba to Viviers. We do not know where exactly, but we know that the bishops stayed near the cathedral until the 14th century.
The first bishop's palace still exists but has been enlarged. Originally it was a tower that housed a guard room, the bishop's room, the archives and the prison. After the 14th century they no longer lived in Viviers, but in their castles: Largentière, St Martin, Donzère, and especially Bourg-St-Andéol; they traveled a lot and often stayed in Avignon or Paris.
Also, when in 1723, François Renaud de Villeneuve was designated for the seat of Viviers, the pope, in his "bulles", asked him to reside in his episcopal city. Meanwhile the ancient palace had found a new use as it sheltered the first seminary in 1650.
The construction of a new palace was needed and in 1730 the bishop asked for plans from the famous architect Jean-Baptiste Franque. To finance the construction, the bishop used 20,000 livres inherited from his predecessors. He sold the Château de Largentière for 44,500 livres as well as the houses of Viviers and Bourg for 17,400 livres.
In 1732, the architect wrote to the bishop: "the floor of the house to be built will be prepared in this week, the vine and trees will be torn off". The first stone was laid in 1732 on the land acquired along the Rhône; the stones were taken at the quarry of St Restitut and carried in a cart to the port of the Rhône called the Robinet. The master mason, Claude Projet, directed the works on site.
But then another construction started next door, Pierre de Roqueplane, tax collector of the Vivarais. It bothered the bishop a lot: "I do not find it bad that the building of M. de Roqueplane advance ... but it is not appropriate that it is at the expense of mine! ".
Bishop de Villeneuve settled in 1737 in his palace which was completed in 1741, and again! The right wing which was to contain the chapel, offices and a greenhouse, was never completed. The main building overlooking the main courtyard housed the reception rooms, lounges, dining room, billiard room and also the famous "Italian-style room". On the first floor, offices, study rooms, bedrooms and chapel.
The ground floor of the left wing was reserved for the kitchens, the gallery, the storerooms, and on the first floor were the "galetas" of the servants.
If the episcopal palace was never completed, the gardens were well finished. There were ponds and fountains, an orangery and an alley of one hundred and sixty lime trees leading to a fresh room. The water supply came from the spring of Fombonne which gushed in the mountain and which was channeled until the bishopric.
After the death of Monsignor Morel de Mons, it was Charles de la Font de Savine who was appointed to Viviers in 1779. His biography portrays him as "endowed with exceptional intelligence, good, generous, educated and literate". But at the same time he is accused of "instability of mind, lightness and capricious piety". A great reception took place in the episcopal palace for his enthronement.
The vote of the civil Constitution of the clergy in 1790 introduced 83 departments with their bishopric and the appointment of the bishops was submitted to the vote of the voters of the dioceses and parishes. As a result of the nationalization of church property, clerics became state officials who were paid a salary. The Constituent Assembly required their oath of obedience and those who refused became refractory.
Bishop de Savine, in love with new ideas, joined the feast of the Federation on July 14, 1790 presiding over the ceremony which brought together the population, the National Guards and men under the command of Marshal Jacques Mercoyrol Beaulieu . Then followed by all the vivarois he swore fidelity to the Nation. He took advantage of his office as President of the Electoral Assembly of Ardeche to cover the shelter of refractory priests.
In 1793 he deposited "at the feet of Motherland" all his ornaments, silver vases, episcopal vestments, butt, miter and cross in gold. He left the bishopric then the city. It was planned to make several lots to sell the bishop's palace estimated at 79,845 pounds but impossible to share in equivalent lots. It was thought of turning it into a country house or a factory, but no buyer came forward. Then in 1801 came the Concordat signed between Pope Pius VII and Napoleon Bonaparte who repealed the Civil Constitution of the clergy. The number of bishoprics was reduced to 50 and the departments of Lozère and Ardèche grouped under the leadership of Bishop Chabot in Mende. The bishopric of Viviers no longer existed.