I know "another guy" who made the same mistake. I had to use a ball joint fork to separate the ball joint from the spindle. Surprisingly, I did not destroy the rubber boot in the process.

Before you get to that point, try tapping the ball joint out from above with a tapered bar and a hammer. Whatever you do, don't pound on the threads or you'll have a hell of a time putting the castle nut back on - the ball joint will just sit there and spin if the threads are even a little damaged. You could also try inverting the castle nut on the threads and tapping it, but that will eventually oval the nut if you don't use a block of wood on the nut.