Advanced TypeScript: reinventing lodash.get

TypeScript is able to infer the correct type using properties path

As a part of a backoffice team in Revolut, I have to deal with a lot of complex data structures: customer personal data, transactions, you name it. Sometimes you need to present a value that lies deep inside a data object. To make life simpler, I could use lodash.get which allows me to access a value by its path, and avoid endless obj.foo && obj.foo.bar conditions (though it's not a case anymore after optional chaining had landed).

What is wrong with this approach?

While _.get works perfectly well in runtime, it comes with a huge drawback when used with TypeScript: in a majority of cases, it cannot infer value type, which could lead to various issues during refactoring.

Let’s say a server sends us data with a customer’s address stored this way

type Address = {
  postCode: string
  street: [string, string | undefined]
}
type UserInfo = {
  address: Address
  previousAddress?: Address
}
const data: UserInfo = {
  address: {
    postCode: "SW1P 3PA",
    street: ["20 Deans Yd", undefined]
  }
}

And now we want to render it

import { get } from 'lodash'
type Props = {
  user: UserInfo
}
export const Address = ({ user }: Props) => (
  <div>{get(user, 'address.street').filter(Boolean).join(', ')}</div>
)

Later, at some point, we would like to refactor this data structure and use a slightly different address representation

type Address = {
  postCode: string
  street: {
    line1: string
    line2?: string
  }
}

Since _.get always returns any for path strings, TypeScript will not notice any issues, while code will throw in runtime, because filter method doesn't exist on our new Address object.

Adding types

Since v4.1, which was released in Nov 2020, TypeScript has a feature called Template Literal Types. It allows us to build templates out of literals and other types. Let’s see how it could help us.

Parsing dot-separated paths

For the most common scenario, we want TypeScript to correctly infer value type by a given path inside an object. For the above example, we want to know a type for address.street to be able to early notice the issue with an updated data structure. I will also use Conditional Types. If you are not familiar with conditional types, just think of it as a simple ternary operator, that tells you if one type matches another.

First of all, let’s check if our path is actually a set of dot-separated fields

type IsDotSeparated<T extends string> = T extends `${string}.${string}`
  ? true
  : false
type A = IsDotSeparated<'address.street'> // true
type B = IsDotSeparated<'address'> // false

Looks simple, right? But how could we extract the actual key? Here comes a magic keyword infer which will help us to get parts of a string

type GetLeft<T extends string> = T extends `${infer Left}.${string}`
  ? Left
  : undefined
type A = GetLeft<'address.street'> // 'address'
type B = GetLeft<'address'> // undefined

And now, it’s time to add our object type. Let’s start with a simple case

type GetFieldType<Obj, Path> = Path extends `${infer Left}.${string}`
  ? Left extends keyof Obj
    ? Obj[Left]
    : undefined
  : Path extends keyof Obj
    ? Obj[Path]
    : undefined

type A = GetFieldType<UserInfo, 'address.street'> // Address, for now we only taking a left part of a path
type B = GetFieldType<UserInfo, 'address'> // Address
type C = GetFieldType<UserInfo, 'street'> // undefined

First, we are checking if our passed path matches string.string template. If so, we are taking its left part, checking if it exists in the keys of our object, and returning a field type.

If the path didn’t match a template, it might be a simple key. For this case, we are doing similar checks and returning field type, or undefined as a fallback.

Adding a recursion

Ok, we got the correct type for a top-level field. But it gives us a little value. Let’s improve our utility type and go down the path to the required value.

We are going to:

1. Find a top-level key
2. Get a value by a given key
3. Remove this key from our path
4. Repeat the whole process for our resolved value and the rest of the key until there’s no Left.Right match

export type GetFieldType<Obj, Path> =
  Path extends `${infer Left}.${infer Right}`
    ? Left extends keyof Obj
      ? GetFieldType<Obj[Left], Right>
      : undefined
    : Path extends keyof Obj
      ? Obj[Path]
      : undefined
type A = GetFieldType<UserInfo, 'address.street'> // { line1: string; line2?: string | undefined; }
type B = GetFieldType<UserInfo, 'address'> // Address
type C = GetFieldType<UserInfo, 'street'> // undefined

Perfect! Looks like that’s exactly what we wanted.

Handling optional properties

Well, there’s still a case we need to take into account. UserInfo type has an optional previousAddress field. Let's try to get previousAddress.street type

type A = GetFieldType<UserInfo, 'previousAddress.street'> // undefined

Ouch! But in case previousAddress is set, street will definitely not be undefined.

Let’s figure out what happens here. Since previousAddress is optional, its type is Address | undefined (I assume you have strictNullChecks turned on). Obviously, street doesn't exist on undefined, so there is no way to infer a correct type.

We need to improve our GetField. To retrieve a correct type, we need to remove undefined. However, we need to preserve it on the final type, as the field is optional, and the value could indeed be undefined.

We could achieve this with two TypeScript built-in utility types: Exclude which removes types from a given union, and Extract which extracts types from a given union, or returns never in case there are no matches.

export type GetFieldType<Obj, Path> = Path extends `${infer Left}.${infer Right}`
  ? Left extends keyof Obj
    ? GetFieldType<Exclude<Obj[Left], undefined>, Right> | Extract<Obj[Left], undefined>
    : undefined
  : Path extends keyof Obj
    ? Obj[Path]
    : undefined
// { line1: string; line2?: string | undefined; } | undefined
type A = GetFieldType<UserInfo, 'previousAddress.street'>

When undefined is present in the value type, | Extract<> adds it to the result. Otherwise, Extract returns never which is simply ignored.

And this is it! Now we have a nice utility type that will help to make our code much safer.

Implementing a utility function

Now that we taught TypeScript how to get correct value types, let’s add some runtime logic. We want our function to split a dot-separated path into parts, and reduce this list to get the final value. The function itself is really simple.

export function getValue<
  TData,
  TPath extends string,
  TDefault = GetFieldType<TData, TPath>
>(
  data: TData,
  path: TPath,
  defaultValue?: TDefault
): GetFieldType<TData, TPath> | TDefault {
  const value = path
    .split('.')
    .reduce<GetFieldType<TData, TPath>>(
      (value, key) => (value as any)?.[key],
      data as any
    );
  return value !== undefined ? value : (defaultValue as TDefault);
}

We have to add some ugly as any type castings because

1. intermediate values could indeed be of any type;
2. Array.reduce expects the initial value to be of the same type as a result. However, it's not the case here. Also, despite having three generic type parameters, we don't need to provide any types there. As all generics are mapped to function parameters, TypeScript infers these upon the function call from the actual values.

Making component type-safe

Let’s revisit our component. In the initial implementation, we used lodash.get which didn't raise an error for a mismatched type. But with our new getValue, TypeScript will immediately start to complain

TypeScript “property filter does not exist” error

Adding support for [] notation

_.get supports keys like list[0].foo. Let's implement the same in our type. Again, literal template types will help us to get index keys from square brackets. I will not go step by step this time and instead will post the final type and some comments below.

type GetIndexedField<T, K> = K extends keyof T 
  ? T[K]
  : K extends `${number}`
    ? '0' extends keyof T
      ? undefined
      : number extends keyof T
        ? T[number]
        : undefined
    : undefined
type FieldWithPossiblyUndefined<T, Key> =
  | GetFieldType<Exclude<T, undefined>, Key>
  | Extract<T, undefined>
type IndexedFieldWithPossiblyUndefined<T, Key> =
  | GetIndexedField<Exclude<T, undefined>, Key>
  | Extract<T, undefined>
export type GetFieldType<T, P> = P extends `${infer Left}.${infer Right}`
  ? Left extends keyof T
    ? FieldWithPossiblyUndefined<T[Left], Right>
    : Left extends `${infer FieldKey}[${infer IndexKey}]`
      ? FieldKey extends keyof T
        ? FieldWithPossiblyUndefined<IndexedFieldWithPossiblyUndefined<T[FieldKey], IndexKey>, Right>
        : undefined
      : undefined
  : P extends keyof T
    ? T[P]
    : P extends `${infer FieldKey}[${infer IndexKey}]`
      ? FieldKey extends keyof T
        ? IndexedFieldWithPossiblyUndefined<T[FieldKey], IndexKey>
        : undefined
      : undefined

To retrieve a value from a tuple or array, there’s a new GetIndexedField utility type. It returns tuple value by a given key, undefined if the key is out of tuple range, or element type for regular array. '0' extends keyof T condition checks if a value is a tuple, as arrays don't have string keys. If you know a better way to distinguish a tuple and an array, please let me know.

We are using ${infer FieldKey}[${infer IndexKey}] template to parse field[0] parts. Then, using the same Exclude | Extract technique as before, we are retrieving value types respecting optional properties.

Now we need to slightly modify our getValue function. For the sake of simplicity, I will replace .split('.') with .split(/[.[\]]/).filter(Boolean) to support new notation. That's probably not an ideal solution, but more complex parsing is out of the scope of the article.

Here’s the final implementation

export function getValue<
  TData,
  TPath extends string,
  TDefault = GetFieldType<TData, TPath>
>(
  data: TData,
  path: TPath,
  defaultValue?: TDefault
): GetFieldType<TData, TPath> | TDefault {
  const value = path
    .split(/[.[\]]/)
    .filter(Boolean)
    .reduce<GetFieldType<TData, TPath>>(
      (value, key) => (value as any)?.[key],
      data as any
    );
  return value !== undefined ? value : (defaultValue as TDefault);
}

Conclusion

Now we not only have a nice utility function that improves code type safety, but also a better understanding of how to apply template literal and conditional types in practice.

I hope the article was helpful. Thank you for reading.

All code is available at this codesandbox