Loading for pure accuracy and loading for reliability are two different things. I'm not saying you cant have an accurate reliable load but when shooting at targets if you stick a bullet in the barrel or a round wont chamber due to being slightly tight you aren't out anything but time. Having the same problem in the field can cost you a missed shot or trip back home to clear the problem so err on the reliability side. If you do elect to neck size at least make sure each round you intend to hunt with will function thru your rifle before heading out, actually its a good practice with any of your reloads.
For powder the 4350 is hard to beat and a good all round bullet is the Sierra 85g. bthp. I use it in a 6mm Remington and also in a 6mm br xp-100 pistol and they both have taken deer cleanly with one shot.