Home/ Software Testing / Smart Home
 
  

Automated Email Testing with Nightwatch and MailTrap

September 27, 2022

Nightwatch is a browser automation test framework built on top of the Selenium Webdriver library popularly used for automating end-to-end testing in web browsers such as Chrome, Firefox, Safari, and Edge. Sometimes these scenarios trigger conditions that have to be verified outside the reach of browser automation such as checking emails.

Issues one may run into checking emails include

  • Many web-based email services have measures in place to prevent misuse by bots like CAPTCHAs, 2FA, or client checks.
  • Login management for QA team for shared email accounts is problematic
  • Network security policies preventing use of internal SMTP servers
  • Send limits
  • Finite email address combinations

MailTrap is a service that provides an email sandbox one can programmatically check in their automated browser tests using a plugin for Nightwatch, nightwatch-mailtrap, that this article will discuss how to use if one were stuck trying to figure out how to test emails with selenium automation frameworks like Nightwatch.

Email Test Assertion Scenarios

Many things can go wrong with email delivery which makes testing it important.

Popular email testing scenarios

  • Did the intended recipient receive the email?
  • Did only the intended recipient receive the email?
  • Did the subject and body content come through complete, formatted, and localized (if applicable)?

In addition, if one is testing parts of a site that includes user account creation it may require following a link in an email to complete the registration process.

Trouble Automating These Email Scenarios

Email tends to be under tight security policies within most organizations as well as the email services themselves. Corporate IT policy may have onerous restrictions on outbound email access or require valid accounts tied to specific employees which makes testing from a continuous build system impractical. In addition, testers shouldn't have to create logins on external platforms like Gmail or Yahoo to verify email delivery. What if someone leaves the company? Further, those systems usually have steps in place to prevent misuse by bots which means those tests can't be automated in your tests.

MailTrap Email Sandbox as a Solution

MailTrap let's one work around these sort of problems by providing an email sandbox one can programmatically check using their email API. MailTrap let's one create one or more inboxes to receive the emails on. One can replace their test environment's outbound email SMTP settings with the special ones provided by MailTrap that correlate to those inboxes. All outbound messages To and From the system under test will end up in the inbox to be verified with API calls to the MailTrap service (or web portal).

// C# Example Sending Mail via MailTrap
static void Main(string[] args)
{
  var client = new SmtpClient("smtp.mailtrap.io", 2525)
  {
    // The credentials correspond to your particular MailTrap inbox
    Credentials = new NetworkCredential("{username}", "{password}"),
    EnableSsl = true
  };
  client.Send("from@example.com", "to@example.com", "Hello world", "testbody");
}

So one can be as creative as they want using any From or To email address and it would end up in the MailTrap inbox associated with the provided network credentials. After, one can use the MailTrap API to programmatically check the emails for use in their tests.

Using MailTrap as a stand-in or sandbox for one's actual corporate SMTP would bypass any network policy restrictions on access or email address validity in one's tests. The MailTrap API resolves the blocks against test automation (automated browser blocks, CAPTCHAs, 2FA requirements), or sharing and management of real email account credentials within the development team.

This next section will cover how to put it all together with Nightwatch test assertions using the nightwatch-mailtrap plugin so one doesn't have to work with the MailTrap API directly.

Email Assertions using the Nightwatch MailTrap Plugin

nightwatch-mailtrap adds additional assertions to the native asserts in Nightwatch.

  • emailSubjectContains
  • emailBodyContains
  • expectedInboxCount

and a command that will return the first link out of the email body.

  • getLinkFromEmail

Here is an example test with email assertions

module.exports = {
  'should get messages': async (browser) => {
    const inboxId = browser.globals.mailtrap.mailboxId;


    // Ability to grab the link URL out of the message body
    let url = await browser.getLinkFromEmail(inboxId);
    // Example navigating the browser to the URL from the message body link
    browser.url(url);


    // Test inbox contains 5 emails
    browser.assert.expectedInboxCount(5, inboxId);


    // Test inbox has 1 email if subject, to_email, or to_name matches "hello"
    browser.assert.expectedInboxCount(1, inboxId, 'hello');


    // Test that the first email message containing Welcome has 
    // "Bienvenue à Nightwatch" in the message body
    browser.assert.emailBodyContains(
      'Bienvenue à Nightwatch',
      inboxId,
      'Welcome'
    );


    // Assert the text of the first email matching 'latest tests' matches 
    // 'The latest tests delivered...'
    browser.assert.emailSubjectContains(
      'The latest tests delivered straight to your inbox',
      inboxId,
      'latest tests' // optional search filter
    );
  },
};

Installation

  1. Start from a working Nightwatch 2.0 installation
  2. From a command line in the test project run npm install nightwatch-mailtrap --save-dev
  3. In the nightwatch.conf.js file in your test project append nightwatch-mailtrap to the plugins property array.
// nightwatch.conf.js
module.exports = {
  ...
  src_folders: ['test'],


  plugins: ['nightwatch-mailtrap'],
  ...
}
  1. Add your MailTrap account settings (from the MailTrap portal under your user settings) to the Nightwatch configuration inside globals.
//nightwatch.conf.js
test_settings: {
  default: {
    launch_url: 'http://localhost',
    desiredCapabilities: {
      browserName: 'chrome',
    },


    globals: {
        mailtrap: {
          apiToken: '${MAILTRAP_API_TOKEN}',
          mailboxId: '${MAILTRAP_MAILBOX_ID}',
        },
    },
  }
}

This is what the plugin will use to authenticate with the MailTrap API. See the README for more detailed installation instructions.

Usage Ideas

Here are some ideas where one might mix the nightwatch-mailtrap assertions in a Nightwatch test flow.

User Registration Flow

This test submits a new user registration form. The form generates an email with a link they must follow to complete the user registration. It uses getLinkFromEmail to get the link from the email, passes it back to Nightwatch to navigate to it, and then verifies the page it navigates to indicates registration was a success (thanks the user for signing up).

'Valid user registration is successful': async (browser) => {
  // Submit new user registration form
  browser.page.userRegistration()
    .submitNewUserForm('John', 'Doe');
  // Get link from generated new user verification email
  const url = await browser.getLinkFromEmail(
    browser.globals.mailtrap.mailboxId,
    'Verify your account');
  // Navigate to verification link from email in your browser
  browser.url(url);
  browser.expect.element('#message').to.equal('Thank you for signing up');
},

Correct Recipients Received Email

This test is verifying after finalizing a payroll in the UI for a salary group (SalaryGroup2), that an email with subject "Your Pay Statement is Ready" is sent to Mary and Ted @testco.com but not Brad because he is not in that salary group.

'Payroll emails sent to correct recipients': async (browser) => {
  // Complete Payroll
  browser.page.payroll()
    .finalizePayroll('SalaryGroup2');
  // Verify only users in SalaryGroup2 (Mary and Ted) received pay ready email
  browser.assert.expectedInboxCount(
    1,
    browser.globals.mailtrap.mailboxId,
    'mary@testco.com'
  );
  browser.assert.emailSubjectContains(
    'Your Pay Statement is Ready',
    browser.globals.mailtrap.mailboxId,
    'mary@testco.com'
  );
  browser.assert.expectedInboxCount(
    1,
    browser.globals.mailtrap.mailboxId,
    'ted@testco.com'
  );
  // Brad isn't in SalaryGroup2
  browser.assert.expectedInboxCount(
    0,
    browser.globals.mailtrap.mailboxId,
    'brad@testco.com'
  );
},

Example test output

Running Payroll emails sent to correct recipients:
────────────────────────────────────────────────────────────────
  √ Testing if inbox id "*redacted*" contains exactly 1 email
  with a subject, to_email, or to_name matching search
  mary@testco.com (222ms)

Conclusion

Mailtrap's ability to act as a sandbox or stand-in for one's real mail servers allows test engineers to be more creative in their email testing and test scenarios previously not fully coverable through test automation. By extending Nightwatch with the nightwatch-mailtrap plugin test engineers can leverage those capabilities directly inside their existing Nightwatch selenium tests. This allows them to use the MailTrap API in the context of familiar Nightwatch assertions and commands inside their tests.

If you are a Nightwatch beginner, be sure to watch my Software Testing Playlist.

Please share the link to this article if you enjoyed it and if you have any questions or comments, please reach out through my social links below 👇

Photo of David Mello

David Mello

Breaker of software, geek, meme enthusiast.

 

DavidMello.com © 2023