All I know is my short horses.. a 14.1hh possibly a 800lbs, refined built arab mare can carry me (230 lbs) plus tack for 8 hours on rugged trails. granted she is mature and finished growing (she is 12 this year i think).
My old arab gelding, Gally, was 14.1, stockier then my mare and carried me until i retired him. Granted I went from 165 to 220 lbs in that time frams (nearly 20 yrs). Never lame, unsound, whatever.
My refined 14.2hh arab gelding, again about 800lbs, can carry my butt and my tack on a 25 mile endurance ride.. in 4 hours. Never been unsound, or lame.. or sore in any way.
I think your guy will be fine once he is finished maturing.
Again I weigh in at 230lbs. Not a measly 160lbs.

Your like half my weight. I dont think your guy will have a problem.