He was the son of Matthew Fortescue, 2nd Baron Fortescue, younger half-brother of Hugh Fortescue, 1st Earl of Clinton (1696–1751), 1st Baron Fortescue and 14th Baron Clinton.

Residences

Earl Fortescue's residences were as follows:

Castle Hill, Filleigh, North Devon.
Ebrington Manor, Gloucestershire.
Weare Giffard Hall, Devon.

Marriage and progeny

Lord Fortescue married Hester Grenville (1767–1847), daughter of the Prime Minister George Grenville, on 10 May 1782. They had nine children:

Lady Hester Fortescue (d. 1873), married Peter King, 7th Baron King and had issue.
Hugh Fortescue, 2nd Earl Fortescue (1783–1861)
Captain Hon. George Mathew Fortescue (1791–1877), married Lady Louisa Ryder, daughter of Dudley Ryder, 1st Earl of Harrowby and had issue.
Lady Mary Fortescue (15 September 1792, Filleigh, Devon – 12 August 1874, London). Married 15 February 1823 to Sir James Hamlyn Williams of Edwinsford, Carms., and Clovelly, Devon. Buried at Talley, Carms., in the family vault at her special request).
Rev. Hon. John Fortescue (1796–1869)
Lady Elizabeth Fortescue (1801–1867), married William Courtenay, 11th Earl of Devon and had issue.
Lady Catherine Fortescue (1787 – 20 May 1854), said to have been deaf and dumb. She married in 1820 (as his second wife) her lifelong friend Hon. Newton Fellowes (1772 – January 1854), of Eggesford House, Devon, who became in the last year of his life 4th Earl of Portsmouth.They had issue, 1 son (the 5th Earl of Portsmouth b. 1825, from whom all later earls are descended) and three daughters.Her husband's two sons by his first wife both died young and/or unmarried before their father inherited the title.
Lady Anne Fortescue (died 1864)
Lady Eleanor Fortescue (1798–1847), chest tomb in Weare Giffard Church, Devon.