The Pattinson family migrated from the UK to Australia prior to James’ birth. He grew up alongside his Brother Darren who played 1 Test match for England, in Dandenong. James attended Haileybury College in Melbourne.

James was a member of the 2008 Australian Under 19 team that played in Malaysia, following getting home he was picked to make his First Class debut for Victoria in November of that year taking 3 wickets in the first innings. It was in the Summer of 2009/10 that he started to make his mark in the State game, breaking the 40 year record for Victoria’s best figures in a One Day game with 6/48 against NSW. James was selected to make his International debut in April 2011 against Bangladesh, it was his Test debut later in 2011 that drew the headlines taking the first 5 wickets to fall in the New Zealand 2nd Innings completely dismantling the batting line up on his way to claim 5/27 and the man of the match award. James would continue to be plagued by injuries for much of his career before undertaking spinal surgery in New Zealand which gave him a new lease on life, since that surgery he won his way back onto the Cricket Australia central contract list and into the Test team for the 2019 Ashes that Australia retained.