OK, so here's a little tip for any newbie Java programmers: jdbc isn't particularly good at figuring stuff out. If you ask it for 480 million rows (as I just did) and don't tell it anything else, it fetches those rows one at a time. Yes, really. It's quite happy to fetch them 10,000 at a time if you suggest that. And, obviously, the latter strategy will be a lot faster, but you have to tell it.
My extract is still slower than I'd like, but at least it's run time is now on the order of magnitude I was expecting (it's pulling about a million rows a minute - I was hoping for twice that, but we're getting there).
No comments:
Post a Comment