The Perils of Jest Snapshot Testing

07 Jan 2019

According to the Jest docs, snapshot tests are useful whenever you want to make sure your UI does not change unexpectedly. That sounds great in theory, but in this post, I will demonstrate why snapshot tests are more trouble than they’re worth.

Say you have a simple button component:

const Button = ({href, children}) => (
  <a href={href}>{children}</a>
)

You can create a snapshot test for it like so:

import React from 'react'
import Button from '../'
import renderer from 'react-test-renderer'

it('renders correctly', () => {
  const tree = renderer
    .create(<Button href="https://myurl.com">My Label</Button>)
    .toJSON()
  expect(tree).toMatchSnapshot()
})

Wow, that was easy. Now any change to the component’s rendered output will cause the test to fail. Sounds great, right?

Well, let me give you a scenario where things go off the rails pretty quick. Say you decide to add a new attribute target but make a typo and call it traget.

const Button = ({href, target, children}) => (
  <a href={href} traget={target}>{children}</a>
)

In your mind, you already expect the test to fail, since by adding a new attribute, you’ve changed the component’s rendered output. In haste, you review the diff but fail to notice the typo. After updating the snapshot you commit your changes and push them to GitHub.

Now you have a snapshot test that passes but makes the wrong assertion about your component’s rendered output. WTF just happened?

Sorting the Catch

When a snapshot test fails, you have to manually review each change and decide whether it’s a bug or a valid change. This process is tedious and prone to human error, especially when there are lots of changes to review.

Files

Snapshot testing is sort of like fishing with a giant net. There’s a certain kind of fish you want to catch (bugs), but you end up catching a lot of other stuff, too (valid changes). The hard part is sorting the catch and deciding what to keep and what to throw back.

The risk is that you update a snapshot thinking a change was valid when it was in fact a bug.

Things get worse if you use jest-styled-components because it stores your component’s style rules with the snapshot. That means having to review every line of CSS that changes, too.

Snapshot Fatigue

Snapshot tests are easy to create, requiring almost no forethought. That’s because they place all of the cognitive burden on the reviewer.

After a while, engineers begin to experience something I call snapshot fatigue and start blindly updating failed snapshots without reviewing them. Once that happens, your snapshot tests are pretty much useless.

Do You Even Need a Snapshot?

A good test should prevent you from accidentally breaking your component’s API. By creating a snapshot test, you’re essentially declaring that your component’s entire rendered output is part of its API and should never change. In some cases that might be what you want but usually a component has some specific functionality that you want to test, and freezing its entire rendered output makes it a pain to refactor and add new features.

Alternatively, you can identify the elements of your component’s UI that are critical to its behaviour and test those specifically.

import React from 'react'
import Button from '../'
import {render, getByText} from 'react-testing-library'
import 'react-testing-library/cleanup-after-each'
import 'jest-dom/extend-expect'

it('renders correctly', () => {
  const {container} = render(<Button href="https://myurl.com" target="_blank">My Label</Button>)
  const button = getByText(container, 'My Label')

  expect(button).toHaveAttribute('target', '_blank')
  expect(button).toHaveAttribute('href', 'https://myurl.com')
})

The test above is explicit. It verifies that the link contains the correct text and that its href and target values are correct. So long as these values don’t change, the component should function as expected.

This method requires a little more forethought but eliminates the burden and risk of manually reviewing snapshot diffs over time. It also allows you to refactor the component to your heart’s content so long as those specific values don’t change.

Discuss on Twitter • Edit on GitHub

Keep Reading