In August 1452 James attempted to avenge his brother's murder by the King, but was then reconciled. In 1454 he joined the Duke of York (then in rebellion against King Henry VI), accused the King of his brother's murder, and defied him, with some 40,000 men. However, owing to the desertion of Lord Hamilton, his men dispersed, and he fled into Annandale. On 12 June 1455 he was attainted, and all of his honors were forfeited.
The power of Douglas was now completely overthrown. The usual forfeitures followed in June 1455 of the earl, his mother, Beatrix, and his brothers. The act of attainder (Act Parl. ii. 75) recites the treasons, and shows how extensive the conspiracy of the Douglases had been. From Lochindorb and Darnaway in the north, to Thrieve in Galloway, they had fortified all their castles against the king, and from them they had made raids wasting the king's lands with fire and sword. Ettrick Forest was now annexed to the crown, and the other estates of the Douglases divided among the chief supporters of the king. Several families rose to greatness out of the ruin of the Douglases. One of their own kindred, George, fourth earl of Angus, was created Lord of Douglas, and a second line of Angus-Douglases almost rivalled the first. Another Douglas, James of Dalkeith, was made Earl of Morton.
He was invested as a Knight, Order of the Garter (K.G.) before 22 April 1463. In 1482 the Scottish Parliament issued a reward for his arrest. In July 1484 he joined the Duke of Albany in an invasion of Scotland, but was defeated and taken prisoner, and sentenced to imprisonment.
Source www.thepeerage.com from The Dictionary of National Biography, Oxford Univ. Press
http://onlinebooks.library.upenn.edu/webbin/metabook?id=dnb