Forcing SimpleTest to use the live database

SimpleTest, the test suite that Drupal started using, and then improved upon, has primarily been used to test modules in their own little sandbox, unaffected by the outside world, user data, or client-desired tweaks. This is perfectly fine when you're working on a controlled piece of code, like a module intended for release. When you're building a client site, however, you often have a much more ephemeral set of quality assurances to make: that this CCK node has seven fields, that this field doesn't show up for that particular user role, that "Body" has been renamed to "Description", or that the front page display has a certain set of blocks.

Each of those case scenarios all involve changes to the "in-progress" database, the one where the client is adding content, or Views are being configured, or blocks are being made and placed, et cetera. Since SimpleTest's default goal is to start fresh for each particular test method, creating new database tables that have no content in them, you'd never be able to test any of the tweaks the client wants on their live site. If someone accidentally deleted a field from a previously perfect content type, SimpleTest wouldn't catch it.

Thankfully, we can fix this by overriding the setUp() and tearDown() methods that our test classes inherit from DrupalWebTestCase - these methods normally handle the creation of the fake database and the cleanup of any fake created data. (Note: this assumes you're using SimpleTest 6.x-2.x or Drupal 7: earlier versions will not work.)

class FunctionalGenericTestCase extends DrupalWebTestCase {
  function getInfo() {
    return array(
      'name' => t('Generic functionality'),
      'description' => t('Generic user functionality tests for custom code.'),
      'group' => t('Trellon Development'),
    );
  }

  // for our functional testing, we want to test the pages and code that
  // we've been generating in the real database. to do this, we need to
  // ignore SimpleTest's normal fake database creation and fake data
  // deletion by overriding it with our own setUp and tearDown. NOTE that
  // if we make our own fake data, we're responsible for cleaning it up!
  function setUp() {
    // support existing database prefixes. if we didn't,
    // the prefix would be set as '', causing failures.
    $this->originalPrefix = $GLOBALS['db_prefix'];
  }
  function tearDown() { }

  // ensure items from mocks exist.
  function testFrontpageChanged() {
    $this->drupalGet('');
    $this->assertNoText(t('Welcome to your new Drupal website!'),
      t('Default Drupal front page has been changed.'));
  }

  // check for all end-user fields.
  function testContentTypeFields() {
    $this->drupalLogin((object)array('name' => 'authed', 'pass_raw' => 'ahem'));

    $this->drupalGet('node/add/story');
    $this->assertNoRaw('Body: ',
      t('"Body" renamed to "Story" per content-type-outline.pdf (2009-02-18).'));
  }
}

As suggested in the comments, by overriding tearDown() we are now responsible for cleaning up any fake data that our tests create. To lessen the effects of this, we plan to create a number of standard test users - one for each client-required user role - that we can use drupalLogin() to become, as opposed to running a drupalCreateUser with a custom set of permissions (where we would then be responsible for deleting the created user and the role). We don't see this as anything too upsetting: these types of tests are all per-client anyways, so while we hope for some reuse (such as the useless default front page test above), the benefits of using SimpleTest for functionality tests such as this outweigh them.

Since the setUp() and tearDown() methods are per-class, we still have the ability to test any custom modules (or complex algorithms, etc.) in a fresh/sandbox environment - we'd just define a new testing class and leave out the overrides.

Note that this approach still emphasizes data and structure over actual display: tests would happily claim that Body has been renamed to Story, but wouldn't be able to tell us that an errant piece of CSS has caused that entire input field to be hidden from view. A human would still have to manually eyeball display issues (caused by CSS, etc.), as well as test any JavaScript functionality.