if that the hammer you are using I can see why it took a few hours. a few smacks on the knuckle with good sized hammer and the taper should vibrate right loose. I can give you some tips on changing that ball joint with no press or other special tools if you need it.