Iain Poulson wrote an article on automated testing for WordPress developers. I’ve used Selenium in the past and we would run test suites before releases as a sanity check.