Not saying this is right, but I have installed several BJ8 starters and never "caulked" the nut you refer to. It is too tight (IMO) to rattle. motorre-assembly!
Mine I have to back off nearly a quarter of a turn to get the split pin in. Suppose I could try and wrench it on a bit tighter, to get to the next slot. -- Alan F Cross BSc LRPS - Heathfield Studios
Sometimes you can add some washers/shims of the right thickness under a nut to get the nut castellations and the split pin hole to line up closer. At least then you don't have to over- or undertorqu