I would cut the ladder in two pieces half as long and I also would replace the handles with just one brush (5 sides nodraw, 1 side a masked texture like general_industrial/ironbar or algiers/railing_natestah both turned by 90 deg).
This error is commonly produced - like Alcoholic said - by a brush's face (here the inner face of the long vertical ladder pieces)with too many vertices touching (all the little horizontal brushes), and above should help.
If you make the rails wider than the rungs it will fix it. I like using 2 X 4 rails and 2 X 2 rungs (units). So, looking from the side, the rails are 4 units wide and the rungs are 2 units wide and centered on the rails.