Partial matching test assertions

When dealing with complex data structures, such as Arrays of Objects or nested Objects, we don't always need to assert the whole data structure comprehensively. Sometimes we only need a subset of the entire structure to describe the relevant assertions for a particular test. Even more, some fields might be non-deterministic and, therefore impossible to assert.

This blog post explores two different approaches to describe partial test assertions:

1. Omitting insignificant data
2. Picking only relevant data

Please note that the code examples in this post use the Jest and Vitest APIs, which are almost similar. However, the ideas and the approaches are universal.

To exemplify the two approaches, let's consider the following case study:

• we have Users in our application;
• Users can purchase a Subscription for themselves;
• Users can also purchase Extra Members for someone else;
• Users can assign Members to benefit from the Subscription;

For all the examples below, we'll write various tests for a function called getUserProfile() having the following signature:

declare function getUserProfile(userId: string): Promise<UserProfile>;

type UserProfile = User & {
    members: Array<User>,
    availableExtraMembers: number,
};

Besides returning the User model, it will also aggregate a subset of the Subscription information, including the purchased and assigned Extra Members. In other words, changing Subscriptions or Extra Members has side effects on the UserProfile.

Omit insignificant data

When testing the primary effect of a function, ideally we'd want to assert the entire data structure. However, some fields might be cumbersome, if not impossible to determine.

Let's look at one example of testing the direct effect of getUserProfile(). For simplicity, we won't use any Subscriptions or Extra Members in this test.

test("getUserProfile() should return aggregated data", async () => {
  // arrange
  const user = await addUser();
  // act
  const profile = await getUserProfile(user.id);
  // assert
  expect(profile).toEqual({
    id: user.id,
    members: [],
    availableExtraMembers: 0,
    isActive: true,
    createdAt: expect.any(String),
  });
});

In the example above, createdAt is automatically generated by the implementation. The exact value is impossible to determine unless we stub the Date constructor. However, since the exact date is irrelevant for the actual test, we can easily assert that we "expect any string".

Some use cases where this approach is handy include:

• Date objects, which are not relevant for the test.
• Non-deterministic data, such as time or timezones, which could return unexpected values when running in CI.
• UUIDs or foreign keys not relevant for the test. They are usually deterministic, but sometimes require additional queries which might not bring any added value.
• Internally generated data might be impossible to assert unless we expose and stub the internal logic. If the logic is important to test, we can add unit tests for that particular code.

It should be noted that there are other matchers that might be useful, depending on the context:

• expect.any() can receive any constructor, such as Number, Boolean, Array, Function, Object, or a particular Class.
• expect.toBeOneOf() is helpful when dealing with non-deterministic nullable types. To be noted that this is not a built-in matcher. It's only available via jest-extended.
• expect.anything() for rare cases where you're dealing with unknown types.

Pick only relevant data

However, having a lot of tests that assert the entire data structure comprehensively will require updates whenever we change the structure. Ideally, we only need a few comprehensive tests. The majority of them should only assert test-specific data. One such category of tests includes side effects.

To begin with a simple use case, let's test that buying Extra Members increments the available count on the UserProfile:

test("Buying Extra Members should increment the UserProfile availability count", async () => {
  // arrange
  const user = await addUser();
  await buySubscription({ userId: user.id });
  // act
  await buyExtraMembers({ userId: user.id, count: 1 });
  // assert
  const profile = await getUserProfile(user.id);
  expect(profile.availableExtraMembers).toEqual(1);
});

Considering that we already have the test from the previous section, which asserts the whole data structure, we don't need to repeat ourselves in this test. We can only include the relevant data in the assertion, namely availableExtraMembers.

Now let's move on to a more involved example. Let's test that when assigning Extra Members, the available count on the UserProfile decrements:

test("Assigning Extra Members should decrement the UserProfile availability count", async () => {
  // arrange
  const user = await addUser();
  await buySubscription({ userId: user.id });
  await buyExtraMembers({ userId: user.id, count: 2 });
  // act
  const member1 = await assignExtraMember({ userId: user.id });
  const member2 = await assignExtraMember({ userId: user.id });
  // assert
  const profile = await getUserProfile(user.id);
  expect(profile).toMatchObject({
    members: expect.arrayContaining([
      expect.objectContaining({ id: member1.id }),
      expect.objectContaining({ id: member2.id }),
    ]),
    availableExtraMembers: 0,
  });
});

In the above test, we actually assert 2 side effects:

1. profile.availableExtraMembers decrements once for each assignExtraMember() call;
2. profile.members includes the ids of the users passed to assignExtraMember();

Take note that we're only asserting the data relevant for the test, ignoring the rest of the User and Member information.

The main advantage of this approach is that the test will only fail when the relevant logic will be changed. For example, if getUserProfile() will return additional fields as a consequence of changing the User model, without affecting the side effects, these tests will not fail.

Conclusions

From personal experience, at least some of the tests should be comprehensive by asserting entire data structures, even if they are complex and nested. At least some tests should fail when the returned structure changes, regardless if there is an extra field or a missing one. However, most tests should assert only specific data relevant for the test case.

Therefore, both approaches covered in this post are useful, addressing different scenarios:

• Omitting insignificant data is useful when testing primary actions, CRUD operations, aggregated data, etc. This approach should be used whenever we want the test to fail when the structure changes in the source code.
• Picking only relevant data on the other hand is useful when testing secondary actions or side effects. This approach is useful because it will fail the tests only when the relevant data used in assertions changes.