deep_pick

Build License

Simplifies manual JSON parsing with a type-safe API.

No dynamic, no manual casting
Flexible inputs types, fixed output types
Useful parsing error messages

import 'package:deep_pick/deep_pick.dart';

pick(json, 'parsing', 'is', 'fun').asBoolOrThrow(); // true

dart pub add deep_pick

dependencies:
  deep_pick: ^1.3.0

Example

This example demonstrates parsing of an HTTP response using deep_pick. You can either use it to parse individual values of a json response or parse whole objects using the fromPick constructor.


  final response = await http.get(Uri.parse('https://pub.dev/api/packages/deep_pick'));
  final json = jsonDecode(response.body);

  // Parse individual fields (nullable)
  final String? name = pick(json, 'name').asStringOrNull();
  
  // Require values to be non-null or throw a useful error message
  final String version = pick(json, 'latest', 'version').asStringOrThrow();
  
  // Pick deep nested values without parsing all objects in between
  final String? sdk = pick(json, 'latest', 'pubspec', 'environment', 'sdk').asStringOrNull();
  
  
  // Parse a full object using a fromPick factory constructor
  final PackageVersion latest = PackageVersion.fromPick(pick(json, 'latest').required());

  
  // Parse lists with a fromPick constructor
  final List<PackageVersion> versions = pick(json, 'versions')
      .asListOrEmpty((pick) => PackageVersion.fromPick(pick));

  // Parse the values of a JSON object
  final Map<String, String> devDependencies =
      pick(json, 'latest', 'pubspec', 'dev_dependencies')
          .asJsonObjectOrEmpty(mapValue: (pick) => pick.asStringOrThrow());

// Http response model
class PackageVersion {
  const PackageVersion({
    required this.version,
    required this.archiveUrl,
    required this.published,
    this.description,
  });

  final String version;
  final String archiveUrl;
  final DateTime published;
  final String? description;

  factory PackageVersion.fromPick(RequiredPick pick) {
    return PackageVersion(
      version: pick('version').asStringOrThrow(),
      archiveUrl: pick('archive_url').asStringOrThrow(),
      published: pick('published').asDateTimeOrThrow(),
      description: pick('pubspec', 'description').asStringOrNull(),
    );
  }
}

Supported types

`String`

Returns the picked Object as String representation. It doesn't matter if the value is actually a int, double, bool or any other Object. pick calls the objects toString method.

pick('a').asStringOrThrow(); // "a"
pick(1).asStringOrNull(); // "1"
pick(1.0).asStringOrNull(); // "1.0"
pick(true).asStringOrNull(); // "true"
pick(User(name: "Jason")).asStringOrNull(); // User{name: Jason}

`int` & `double`

pick tries to parse Strings with int.tryParse and double.tryParse. A int can be parsed as double (no precision loss) but not vice versa because it could lead to mistakes.

pick(3).asIntOrThrow(); // 3
pick("3").asIntOrNull(); // 3
pick(1).asDoubleOrThrow(); // 1.0
pick("2.7").asDoubleOrNull(); // 2.7

`bool`

Parsing a bool couldn't be easier with those self-explaining methods

pick(true).asBoolOrThrow(); // true
pick(false).asBoolOrThrow(); // false
pick(null).asBoolOrTrue(); // true
pick(null).asBoolOrFalse(); // false
pick(null).asBoolOrNull(); // null
pick('true').asBoolOrNull(); // true;
pick('false').asBoolOrNull(); // false;

deep_pick does not treat the int values 0 and 1 as bool as some other languages do. Write your own logic using .let instead.

pick(1).asBoolOrNull(); // null
pick(1).letOrNull((pick) => pick.value == 1 ? true : pick.value == 0 ? false : null); // true

`DateTime`

Accepts most common date formats such as

ISO 8601 including RFC-3339,
RFC 1123 including HTTP date header, RSS2 pubDate, RFC 822, RFC 1036 and RFC 2822,
RFC 850 including RFC 1036, COOKIE
ANSI C asctime()

pick('2021-11-01T11:53:15Z').asDateTimeOrNull(); // UTC
pick('2021-11-01T11:53:15+0000').asDateTimeOrNull(); // ISO 8601
pick('Monday, 01-Nov-21 11:53:15 UTC').asDateTimeOrThrow(); // RFC 850
pick('Wed, 21 Oct 2015 07:28:00 GMT').asDateTimeOrThrow(); // RFC 1123
pick('Sun Nov  6 08:49:37 1994').asDateTimeOrThrow(); // asctime()

`List`

When the JSON object contains a List of items that List can be mapped to a List<T> of objects (T).

pick([]).asListOrNull(SomeObject.fromPick);
pick([]).asListOrThrow(SomeObject.fromPick);
pick([]).asListOrEmpty(SomeObject.fromPick);

final users = [
  {'name': 'John Snow'},
  {'name': 'Daenerys Targaryen'},
];
List<Person> persons = pick(users).asListOrEmpty((pick) {
  return Person(
    name: pick('name').required().asStringOrThrow(),
  );
});

class Person {
  final String name;

  Person({required this.name});
}

Note 1

Extract the mapper function and use it as a reference allows to write it in a single line again :smile:

List<Person> persons = pick(users).asListOrEmpty(Person.fromPick);

Dart 2.15 and newer also support constructor tear-offs, so Person.fromPick can be a factory constructor. For Dart 2.12–2.14, use .asListOrEmpty((it) => Person.fromPick(it)) when using a factory constructor.

Note 2

pick called in the fromPick function uses the parameter pick, not the top-level function. This is possible because Pick implements the .call() method. This allows chaining indefinitely on the same Pick object while maintaining internal references for useful error messages.

Both versions produce the same result and shows you're not limited to 10 arguments.

pick(json, 'shoes', 1, 'tags', 0).asStringOrThrow();
pick(json)('shoes')(1)('tags')(0).asStringOrThrow();

whenNull

To simplify the asList API, the functions ignores null values in the List. This allows the usage of RequiredPick over Pick in the map function.

When null is important for your logic you can process the null value by providing an optional whenNull mapper function.

pick([1, null, 3]).asListOrNull(
  (it) => it.asIntOrThrow(),
  whenNull: (Pick pick) => 25,
);
// [1, 25, 3]

For the null values of a Map, the same function is called whenNullValue.

Plain lists

asPlainList* returns the items of a List as they are, without a mapper. Nothing is converted and null items are kept.

List<Object?> tags = pick(json, 'tags').asPlainListOrEmpty();

Declare the type of the items to get a typed list.

List<String> tags = pick(json, 'tags').asPlainListOrEmpty();

Every item has to be of that type already. An item of another type throws a PickException that names the item, like expected an item of type String at tags[1]. Use asList* when the items need converting, for example to read 1 as "1".

`Map`

Picking the Map is rarely used, because Pick itself grants further picking using the .call(args) method. Converting back to a Map is usually only used for existing fromMap mapper functions.

pick(json).asMapOrNull<String, dynamic>();
pick(json).asMapOrThrow<String, dynamic>();
pick(json).asMapOrEmpty<String, dynamic>();

To parse the entries instead of casting them, see Mapping the keys and values of a Map.

JSON objects

asJsonObject* returns a Map<String, Object?>, the type fromJson functions expect. No type arguments needed.

pick(json, 'user').asJsonObjectOrNull();
pick(json, 'user').asJsonObjectOrThrow();
pick(json, 'user').asJsonObjectOrEmpty();

final user = User.fromJson(pick(json, 'user').asJsonObjectOrThrow());

A JSON object only has String keys. Entries with any other key are ignored, use asMap* for those maps.

pick({'a': 1, 2: 'two'}).asJsonObjectOrThrow();
// {'a': 1}

Mapping the keys and values of a `Map`

asMap* and asJsonObject* take three optional functions to parse the entries: mapValue, whenNullValue and mapKey.

mapValue

mapValue parses each value, like the mapper of asList* does for the items of a list.

// {"nike": {"id": "421"}, "adidas": {"id": "532"}}
Map<String, Shoe> shoes = pick(json).asJsonObjectOrEmpty(mapValue: Shoe.fromPick);

An error inside a value reports the full location, like adidas.id.

whenNullValue

mapValue ignores entries with a null value, like asList* ignores null items. Use whenNullValue to map them as well, it is the whenNull of maps.

// {"nike": {"id": "421"}, "reebok": null}
Map<String, Shoe?> shoes = pick(json).asJsonObjectOrEmpty(
  mapValue: Shoe.fromPick,
  whenNullValue: (pick) => null,
);
// reebok is kept, with a null value

mapKey

mapKey converts the keys. It receives a pick of the key, so the usual parsers work on it.

// {"421": {"name": "Nike Zoom Fly 3"}, "532": {"name": "adidas Ultraboost"}}
Map<int, Shoe> shoes = pick(json).asMapOrEmpty(
  mapKey: (key) => key.asIntOrThrow(),
  mapValue: Shoe.fromPick,
);

The keys of a JSON object stay Strings, the mapKey of asJsonObject* can rename them. Use asMap* to convert them to another type.

Lists of JSON objects and `Map`s

asJsonObjectList* is the shorthand for a List of JSON objects.

// before
pick(json, 'users').asListOrEmpty((pick) => pick.asMapOrThrow<String, dynamic>());
// after
pick(json, 'users').asJsonObjectListOrEmpty();

List<User> users = pick(json, 'users').asJsonObjectListOrEmpty().map(User.fromJson).toList();

asMapList* does the same for maps with any key type.

pick(data).asMapListOrNull(); // List<Map<dynamic, dynamic>>?
pick(data).asMapListOrThrow<int, String>(); // List<Map<int, String>>
pick(data).asMapListOrEmpty<int, String>();

OrEmpty and OrNull cover the list being absent or not a List. An item that isn't a Map throws a PickException that names the item, null items are ignored.

Custom parsers

Parsers in deep_pick are based on extension functions on the classes Pick. This makes it flexible and easy for 3rd-party types to add custom parsers.

This example parses a int as Firestore Timestamp.

import 'package:cloud_firestore/cloud_firestore.dart';
import 'package:deep_pick/deep_pick.dart';

extension TimestampPick on Pick {
  Timestamp asFirestoreTimeStampOrThrow() {
    final value = required().value;
    if (value is Timestamp) {
      return value;
    }
    if (value is int) {
      return Timestamp.fromMillisecondsSinceEpoch(value);
    }
    throw PickException.fromPick(this, 'expected a Timestamp');
  }

  Timestamp? asFirestoreTimeStampOrNull() {
    if (value == null) return null;
    try {
      return asFirestoreTimeStampOrThrow();
    } catch (_) {
      return null;
    }
  }
}

let

When using a custom type in only a few places, it might be overkill to create all the extensions. For those cases use the let function borrowed from Kotlin to creating neat one-liners.

final UserId? id = pick(json, 'id').letOrNull((it) => UserId(it.asStringOrThrow()));
final Timestamp timestamp = pick(json, 'time')
    .letOrThrow((it) => Timestamp.fromMillisecondsSinceEpoch(it.asIntOrThrow()));

Examples

Reading documents from Firestore

Picking values from a Firebase DocumentSnapshot is usually very selective. Only a fraction of the properties have to be parsed. In this scenario it would be overkill to map the whole document to a Dart object. Instead, parse the values in place while staying type-safe.

Use .asStringOrThrow() when confident that the value is never null and always exists. The return type then becomes non-nullable (String instead of String?). When the data doesn't contain the full_name field (against your assumption) it would crash throwing a PickException.

final DocumentSnapshot userDoc =
    await FirebaseFirestore.instance.collection('users').doc(userId).get();
final data = userDoc.data();
final String fullName = pick(data, 'full_name').asStringOrThrow();
final int? level = pick(data, 'level').asIntOrNull();

deep_pick offers an alternative required() API with the same result. This is useful to make sure a value exists before parsing it. In case it is null or absent a useful error message is printed.

final String fullName = pick(data, 'full_name').required().asStringOrThrow();

Background & Justification

Before Dart 2.12 and the new ?[] operator one had to write a lot of code to prevent crashes when a value isn't set! Reducing this boilerplate was the origin of deep_pick.

String milestoneCreator;
final milestone = json['milestore'];
if (milestone != null) {
  final creator = json['creator'];
  if (creator != null) {
    final login = creator['login'];
    if (login is String) {
      milestoneCreator = login;
    }
  }
}
print(milestoneCreator); // octocat

This example of parses an issue object of the GitHub v3 API.

Today with Dart 2.12+ parsing Dart data structures has become way easier with the introduction of the ?[] operator.

final json = jsonDecode(response.data);
final milestoneCreator = json?['milestone']?['creator']?['login'] as String?;
print(milestoneCreator); // octocat

deep_pick backports this short syntax to previous Dart versions (<2.12).

final milestoneCreator = pick(json, 'milestone', 'creator', 'login').asStringOrNull();
print(milestoneCreator); // octocat

Still better than vanilla

But even with the latest Dart version, deep_pick offers fantastic features over vanilla parsing using the ?[] operators:

1. Flexible input types

Different languages and their JSON libraries generate different JSON. Sometimes ids are String, sometimes int. Booleans are provided as true or with quotes as String "true". The meaning is the same but from a type perspective they are not.

deep_pick does the basic conversions automatically. By requesting a specific return type, apps won't break when a "price" usually returns double (0.99) but for whole numbers int (1 instead of 1.0).

pick(2).asIntOrNull(); // 2
pick('2').asIntOrNull(); // 2 (Sting -> int)

pick(42.0).asDoubleOrNull(); // 42.0
pick(42).asDoubleOrNull(); // 42.0 (double -> int)

pick(true).asBoolOrFalse(); // true
pick('true').asBoolOrFalse(); // true (String -> bool)

2. No RangeError for Lists

Using the ?[] operator can crash for Lists. Accessing a list item by index outside of the available range causes a RangeError. You can't access index 23 when the List has only 10 items.

json['shoes']?[23]?['id'] as String?;

// Unhandled exception:
// RangeError (index): Invalid value: Not in inclusive range 0..10: 23

pick automatically catches the RangeError and returns null.

pick(json, 'shoes', 23, 'id').asStringOrNull(); // null

3. Useful error message

Vanilla Dart returns a type error because null is not a String. There is no information available which part is null or missing.

final milestoneCreator = json?['milestone']?['creator']?['login'] as String;

// Unhandled exception:
// type 'Null' is not a subtype of type 'String' in type cast

deep_pick shows the exact location where parsing failed, making it easy to report errors to the API team.

final milestoneCreator = pick(json, 'milestone', 'creator', 'login').asStringOrThrow();

// Unhandled exception:
// PickException: expected a non-null value at milestone.creator.login, but it is absent
//
//   query   milestone.creator.login
//                     ~~~~~~~ null has no key "creator"
//   at      milestone = null
//   hint    Use asStringOrNull() when the value may be null/absent at some point (String?).

The query row shows the full requested path with a marker under the segment that could not be followed, and the at row shows the data at the deepest reachable location.

Notice the distinction between "absent" and "null" when you see such errors.

"absent" means the path could not be followed to the end, e.g. the key isn't found in a Map, a List has no item at the requested index, or a null value blocked the way down
"null" means the path was fully followed and the value at the end is actually null

Redacting values in error messages

Error messages show the data at the location parsing failed. That's great during development, but when API responses carry personal data those messages may end up in crash reporters and logs. Add .redactValues() at the root of a pick chain to mask all values while keeping Map keys and types visible. Keys count as schema, so the keys of a Map that is keyed by data, like {"jane@example.com": {...}}, stay visible too:

pick(response).redactValues().letOrThrow((pick) => User.fromPick(pick));

// PickException: expected a non-null value at shoes[0].name, but it is absent
//
//   query   shoes[0].name
//                    ~~~~ no such key
//   at      shoes[0] = Map with keys "id", "size"

Pass enabled to decide at runtime, for example to keep the values in debug builds:

pick(response).redactValues(enabled: kReleaseMode).letOrThrow((pick) => User.fromPick(pick));

4. Null is default, crashes intentional

Parsing objects from external systems isn't type-safe. API changes happen, and it is up to the consumer to decide how to handle them. Consumer always have to assume the worst, such as missing values.

It's so easy to accidentally cast a value to String in the happy path, instead of String? accounting for all possible cases. Easy to write, easy to miss in code reviews.

Forgetting that null could be a valid return type results in a type error:

json?['milestone']?['creator']?['login'] as String;
//                                      ----^
// Unhandled exception:
// type 'Null' is not a subtype of type 'String' in type cast

With deep_pick, all casting methods (.as*()) have null in mind. For each type you have to choose between at least two ways to deal with null.

pick(json, ...).asStringOrNull();
pick(json, ...).asStringOrThrow();

pick(json, ...).asBoolOrNull();
pick(json, ...).asBoolOrFalse();

pick(json, ...).asListOrNull(SomeClass.fromPick);
pick(json, ...).asListOrEmpty(SomeClass.fromPick);

Having "Throw" and "Null" in the method name, clearly indicates the possible outcome in case the values couldn't be picked. Throwing is not a bad habit, some properties are essential for the business logic and throwing an error the correct handling. But throwing should be done intentional, not accidental.

5. Map Objects with let

Even with the new ?[] operator, mapping a value to a new Object (i.e. when wrapping it in a domain Object) can't be done in a single line.

final value = json?['id'] as String?;
final UserId id = value == null ? null : UserId(value);

deep_pick borrows the let function from Kotlin creating a neat one-liner

final UserId? id = pick(json, 'id').letOrNull((it) => UserId(it.asStringOrThrow()));

License

Copyright 2019 Pascal Welsch

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

   http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

deep_pick

Example

Supported types

`String`

`int` & `double`

`bool`

`DateTime`

`List`

Note 1

Note 2

whenNull

Plain lists

`Map`

JSON objects

Mapping the keys and values of a `Map`

mapValue

whenNullValue

mapKey

Lists of JSON objects and `Map`s

Custom parsers

let

Examples

Reading documents from Firestore

Background & Justification

Still better than vanilla

1. Flexible input types

2. No RangeError for Lists

3. Useful error message

Redacting values in error messages

4. Null is default, crashes intentional

5. Map Objects with let

License

Libraries

deep_pick package

deep_pick

Example

Supported types

String

int & double

bool

DateTime

List

Note 1

Note 2

whenNull

Plain lists

Map

JSON objects

Mapping the keys and values of a Map

mapValue

whenNullValue

mapKey

Lists of JSON objects and Maps

Custom parsers

let

Examples

Reading documents from Firestore

Background & Justification

Still better than vanilla

1. Flexible input types

2. No RangeError for Lists

3. Useful error message

Redacting values in error messages

4. Null is default, crashes intentional

5. Map Objects with let

License

Libraries

deep_pick package

`String`

`int` & `double`

`bool`

`DateTime`

`List`

`Map`

Mapping the keys and values of a `Map`

Lists of JSON objects and `Map`s