It’s comforting to believe you can rely on promises. But anonymous JavaScript test functions… yeesh, flaking is literally part of their DNA.

With apologies to any Placebo fans, this post is actually about automated tests that rely on asynchronous behaviour, and therefore on Promise fulfillment. I’m working with TypeScript, Ionic & Angular and the Protractor test runner, and will give an example based on these. But I think there’s a range of software stacks where async tests could hit the same issues.

Some posts introducing end-to-end testing in new Ionic versions implicitly acknowledge that to work with some components, you need your test to work through a chain of promises and nested then()s. And some older writing sensibly suggests writing custom helpers that return their own promises, to avoid nesting deeper and deeper for complex tests, reduce repetition and keep things readable.

However neither of these seems to flag the killer gotcha I found with promises in Protractor tests: if your assertions occur within the success callback (then(() => {...})), as they must to happen after the async part is done, you will never find out if the promise fails. The assertions will simply not be executed and your test will pass, even if everything is broken.

This is a really good way to write a big, reassuring-looking test suite that does absolutely nothing. Terrifying! Once you spot it, it’s obvious, but I believe this should be in huge writing at the top of every introduction to testing with Protractor.

The solution is simple once you notice the problem, especially if you already abstract async component interactions to helper test methods. Using TypeScript and native Promises, here’s an example of the pattern I’ve used to fix this for Webful PasswordMaker.

public setIonicToggle(elementName: string, shouldBeChecked: boolean): Promise<boolean> {
  const ionicInput = element(by.css(`ion-toggle[name="${elementName}"]`));

  return new Promise<boolean>(resolve => {
    ionicInput.getAttribute('checked').then((currentValue: string) => {
      const isChecked: boolean = (currentValue === 'true');
      if (isChecked !== shouldBeChecked) {
        browser.waitForAngular();
        ionicInput.click();
      }
      resolve(true);
    });
  });
}

...

it('should test something...', () => {
  ...
  expect(page.setIonicToggle('domain_only', true)).toEqual(true);
  ...
});

We’ve fixed a few things here, compared to nested then()s in the test itself:

• The main test case now has no nesting from promises.
• Protractor automatically waits exactly long enough for the toggle to be checked, and verifies that it’s actually happened rather than leaving a later step to fail as a side effect. If the promise fails, we’ll see a failed assertion at the specific step that went wrong.
• All subsequent assertions are now unconditional, and guaranteed to actually be evaluated.

Better test readability, and no more broken promises!