Using NSFetchedResultsController’s diffable snapshot with UICollectionView’s diffable data source

A look into the intricate details of Core Data NSFetchedResultsController’s new delegate method to work with diffable data sources driving a collection view.

Jul 13, 2022 by Aleksandar Vacić 11 minute read

Last several weeks, we (Radiant Tap) are working on a major app redesign for one of our clients. App is super-data-intensive, combining multiple different data sources into one complex bi-directional scrollable view. Here’s some samples of the design:

Important bits what’s in there:

First section is the horizontally scrollable set of ads, which are fetched from JSON file. Rarely changes but it can.
All the other sections are using data from Core Data store where some of them change very often, every 3-5s.
Second section is horizontally scrollable set of active sports driven by NSFetchedResultsController (FRC for short) which has no sectionNameKeyPath set.
Third section is horizontally scrollable set of matches (events) driven by another custom FRC, again with sectionNameKeyPath not set.
Then we have another FRC loading non-live matches with sectionNameKeyPath set to the sport’s name attribute (so the matches are grouped by sport). There can be 0 or more of these sections.
This is followed by another FRC with live-matches, again grouped per sport. There can also be 0 or more of these sections.

It’s very complex so far but it does not stop there since there’s quite a bit of additional state we need to maintain on top of displaying incoming data. So we have 3 semantically different UI-update triggers:

New ads (dis)appearing in the first section.
Sport/Event changes in Core Data’s context.
Auxiliary state changes thus even though actual underlying Core Data source did not change, the UICV still needs an update.

I’m describing all this to set the stage, to make it clearer why we did things the way we did them and why it did not always work reliably.
And what eventually did.

Modeling disparate data source into one snapshot

Natural choice here in modern iOS is to use UIKit’s compositional layout. Most examples you find online are using simple types like Int, String or UUID as SectionIdentifierType or ItemIdentifierType in the NSDiffableDataSourceSnapshot.

That is not really an option in the complex case like what we have here. You need much more specificity and customisation. enum with associated values is perfect tool for that and here’s (simplified) set of cases we ended up using for the UICollectionViewDiffableDataSource<SectionItem, GridItem> acting as UICV’s data source:

enum SectionItem: Hashable {
	case banners
	case sportspicker
	case featuredMatches(events: [Event])
	case topstandardsport(Sport, events: [Event])
	case toplivesport(Sport, events: [Event])
}

enum GridItem: Hashable {
	case banner(ad: Ad)
	case sport(Sport)

	case featuredMatch(Event)
	case featuredSelection(Selection, isAddedToBetslip: Bool)

	case eventHeader(Event)
	case marketHeader(Game, sectionIndex: Int, rowIndex: Int)
	case selection(Selection, isAddedToBetslip: Bool)
	case message(sectionIndex: Int, rowIndex: Int, message: String)
	case locked(sectionIndex: Int, rowIndex: Int)
}

All these types — Sport, Event, Game, Selection — are Core Data entities thus all are subclasses of NSManagedObject. Ad is a struct. The accompanying snapshot uses the same identifiers, which are typealias-ed for clearer code:

typealias GridSource = UICollectionViewDiffableDataSource<SectionItem, GridItem>

typealias Snapshot = NSDiffableDataSourceSnapshot<SectionItem, GridItem>

Why is this good? Having the full objects as part of the diffable data source makes vending cells and building the layout to house them very straightforward. For example, cell vending:

func cell(collectionView: UICollectionView, indexPath: IndexPath, item: GridItem) -> UICollectionViewCell {
		switch item {
			case .banner(let ad):
				let cell: AdCell = collectionView.dequeueReusableCell(forIndexPath: indexPath)
				cell.populate(with: ad)
				return cell

			case .sport(let sport):
				let cell: SportPickerCell = collectionView.dequeueReusableCell(forIndexPath: indexPath)
				cell.populate(with: sport.icon, caption: sport.name, isSelected: false)
				return cell

			case .featuredMatch(let event):
				let cell: FeaturedEventCell = collectionView.dequeueReusableCell(forIndexPath: indexPath)
				cell.populate(
					with: event,
					isStreaming: bettingContentManager.isEventStreamingOnTV(event)
				)
				return cell

With all this setup, now we need to configure UICV to use GridSource as data source, setup cell vending method and…build a snapshot:

gridSource = GridSource(collectionView: cv, cellProvider: {
	[unowned self] cv, indexPath, gridItem in
	self.cell(collectionView: cv, indexPath: indexPath, item: gridItem)
})

snapshot(animated: false)

So what about snapshot?

The simplest approach here we tried first:

func snapshot(animated: Bool) {
	if collectionView == nil { return }

	var snapshot = Snapshot()
	var sectionIndex: Int = 0

	//	ads
	snapshot = populateSnapshotWithAds(snapshot)
	sectionIndex += 1

	//	sports picker
	snapshot = populateSnapshotWithSports(snapshot)
	sectionIndex += 1

	//	featured matches
	snapshot = populateSnapshotWithFeaturedMatches(snapshot, sectionIndex: sectionIndex)
	sectionIndex += 1

	//	top standard matches (events)
	for section in frcTopStandardEvents?.sections ?? [] {
		snapshot = populateSnapshot(snapshot, withTopStandardSection: section, sectionIndex: sectionIndex)
		sectionIndex += 1
	}

	//	top live matches (events)
	for section in frcTopLiveEvents?.sections ?? [] {
		snapshot = populateSnapshot(snapshot, withTopLiveSection: section, sectionIndex: sectionIndex)
		sectionIndex += 1
	}

	gridSource.apply(snapshot, animatingDifferences: animated)
}

Watch for changes in the data / state. When they happen, call snapshot(animated: true)
That would build entire new full snapshot, for all sections/items.
For Core Data stuff, we simply iterate over each FRC’s sections and fetchedObjects arrays and convert them into SectionItem and GridItem instances, as needed
Call applySnapshot and let UIKit figure-out the differences and render them.

All changes including those from Core Data are simple to observe. For Core Data, laziest possible approach:

extension HomeDataSource: NSFetchedResultsControllerDelegate {
	func controllerDidChangeContent(_ controller: NSFetchedResultsController<NSFetchRequestResult>) {
		snapshot(animated: true)
	}
}

Looks super easy, right? This…actually works very well and in limited testing — “works on my dev machine” kind of testing — it yielded no issues, no crashes.

Core Data crashes and why they happen

Shortly after entering limited internal QA, we started receiving multiple crash reports per tester. They looked like this:

I knew from past experience what a crash in those UICollectionView.m lines meant; the return of dreaded NSInternalInconsistencyException crash, better known as attempt to delete item X1 from section Y which only contains X2 items before the update and similar exception messages.
DiffableDataSource is supposed to deliver us from it thus disheartening to see it roaring back like this.

There’s not much good posts on this topic to be found. One real gem is from Antoine van der Lee who dived deep into this and figured out the reason for the crashes:

During the transition from an old snapshot to a new snapshot, it could be that the data source is asking for a cell for a given index path that not yet exists in the collection of fetched objects.
The fetched objects in an NSFetchedResultsController are updated after the snapshot is applied but a cell can be asked before that transition finishes.
This mainly happens when a transition is animated.

Entire post is worth a detailed and careful read and boils down to this:

You should use that new FRC delegate method that delivers the snapshot of changes FRC has already digested.
Do not use FRC’s fetchedObjects when building your diffable data model since it’s populated after FRC reports snapshot changes.

Second point was confirmed once I naively tried the easy way out: keep my code as it is but use the newer delegate method:

extension HomeDataSource: NSFetchedResultsControllerDelegate {
	func controller(_ controller: NSFetchedResultsController<NSFetchRequestResult>, didChangeContentWith newSnapshot: NSDiffableDataSourceSnapshotReference) {

		let animateChanges = collectionView.numberOfSections ?? 0 > 0
		snapshot(animated: animateChanges)
	}
}

This yielded the same intermittent crashes:

Using FRC-provided sections and objects almost works. But in some edge cases it leads to inconsistent internal state in the UICV data source thus causing a crash. There’s no solution for it with the code approach above; it needs a complete overhaul.

The thing to understand here is that FRC does not have layout or cells to worry about so it’s free to update its sections and fetchedObjects properties whenever it sees fit. You can’t depend on them to be ready in time for your snapshot builder accessing them. You are opening yourself to race conditions which rarely happen but eventually for someone they do.
In our testing it turned out that we can have 100s or even 1000s of successful snapshot driven updates before encountering a crashing condition.

FRC internally works out the inserted/updated/deleted set of changes and the delegate method reports that to you but there are apparently no guarantees that you can use sections and fetchedObjects as soon as you get the delegate call. When you think about it — there must be some good reason you are given a set of NSManagedObjectIDs and not ready-to-use NSManagedObject instances.

Hence why Antoine’s cell-vending method uses NSManagedObjectID as input and then fetches full object from the NSManagedObjectContext when it needs to populate the cell:

let diffableDataSource = UICollectionViewDiffableDataSource<Int, NSManagedObjectID>(collectionView: collectionView) {
    (collectionView, indexPath, objectID) -> UICollectionViewCell? in
    
    guard let object = try? managedObjectContext.existingObject(with: objectID) else {
        preconditionFailure("Managed object should be available")
    }

    let cell = ...
}

We need to change our Section/Grid enum to use objectIDs and hope for the best. But first we need to see what exactly is Core Data giving us.

Demystifying Core Data snapshots

The snapshot’s type used by Core Data delegate method is NSDiffableDataSourceSnapshotReference and Apple says that you should use Swift-ified counterparts:

Avoid using this type in Swift code.
Only use this type to bridge from Objective-C code to Swift code by typecasting from a snapshot reference to a snapshot:
let snapshot = snapshotReference as NSDiffableDataSourceSnapshot<Int, UUID>

In the SwiftLee post, it is type-casted into ...<Int, NSManagedObjectID>. I found both of these to be wrong. Maybe something has changed since 2020 but the true type in my testing turns out to be:

NSDiffableDataSourceSnapshot<String, NSManagedObjectID>

where String value of sectionIdentifier is:

Some (random?) string when FRC’s sectionNameKeyPath is nil.
Attribute value returned as sectionNameKeyPath when key-path is not nil.

Here are examples logged in Xcode console:

[
"d69e6c783a242772974cfc99189691b88e9d37c3 : x-coredata://E43CC0BD-13BD-40F2-B354-BB660E18C80A/Sport/p27",
"d69e6c783a242772974cfc99189691b88e9d37c3 : x-coredata://E43CC0BD-13BD-40F2-B354-BB660E18C80A/Sport/p23"
]
...
[
"Football : x-coredata://E43CC0BD-13BD-40F2-B354-BB660E18C80A/Event/p735",
"Football : x-coredata://E43CC0BD-13BD-40F2-B354-BB660E18C80A/Event/p747",
"Football : x-coredata://E43CC0BD-13BD-40F2-B354-BB660E18C80A/Event/p752",
"Basketball : x-coredata://E43CC0BD-13BD-40F2-B354-BB660E18C80A/Event/p745"]

Now that we know what we are working with, let’s dig in.

Proper marriage of FRC snapshots with UICV diffable data source

First, we update the diffable data model to:

use NSManagedObjectID instead of full Core Data entities
use String sectionIdentifier we are receiving instead of Int sectionIndex (these have some internal use, specific to our layout)

With some other changes specific to our use-case, this was the end result:

enum SectionItem: Hashable {
	case banners
	case sportspicker
	case featuredMatches(events: [NSManagedObjectID])
	case topstandardsport(sportId: Int64, events: [NSManagedObjectID])
	case toplivesport(sportId: Int64, events: [NSManagedObjectID])
}

enum GridItem: Hashable {
	case banner(ad: Ad)
	case sport(NSManagedObjectID)

	case featuredMatch(NSManagedObjectID)
	case featuredSelection(NSManagedObjectID, isAddedToBetslip: Bool)

	case eventHeader(NSManagedObjectID)
	case marketHeader(NSManagedObjectID, sectionIdentifier: String, rowIndex: Int)
	case selection(NSManagedObjectID, isAddedToBetslip: Bool)
	case message(sectionIdentifier: String, rowIndex: Int, message: String)
	case locked(sectionIdentifier: String, rowIndex: Int)
}

Next — apply all recommendations from SwiftLee post, namely:

do not access FRC’s fetchedObjects directly
instead setup each FRC, set its delegate and only then call performFetch()
in the delegate method, call out your snapshot builder passing the snapshot received from Core Data
convert Core Data snapshot into your own and apply differences

func controller(_ frc: NSFetchedResultsController<NSFetchRequestResult>, didChangeContentWith snapshot: NSDiffableDataSourceSnapshotReference) {

	let cds = snapshot as NSDiffableDataSourceSnapshot<String, NSManagedObjectID>

	if frc == frcSports {
		snapshotSports(usingCoreDataSnapshot: cds, animated: true)

	} else if frc == frcFeaturedEvents {
		snapshotFeaturedMatches(usingCoreDataSnapshot: cds, animated: true)

	} else if frc == frcTopStandardEvents {
		snapshotTopStandardEvents(usingCoreDataSnapshot: cds, animated: true)

	} else if frc == frcTopLiveEvents {
		snapshotTopLiveEvents(usingCoreDataSnapshot: cds, animated: true)
	}
}

Key thing we found here is that Core Data snapshot you receive contains entire set of objects that satisfy the FRC’s predicate. It’s not some delta of few changed objects — it’s all of them. That is awesome since we don’t have to deal with merging of existing data and received delta.

Since we have multiple FRCs and some other additional data sources, we can’t just blindly call applySnapshot(...). Doing that will essentially remove all data and display this one section you just received. Correct way is to take full existing snapshot of your data source, delete the relevant section and then replace it with data from received Core Data snapshot.

func snapshotFeaturedMatches(usingCoreDataSnapshot cds: NSDiffableDataSourceSnapshot<String, NSManagedObjectID>, animated: Bool) {
	// start with existing snapshot
	var snapshot = gridSource.snapshot()

	// delete corresponding section
	snapshot.deleteSections(
		//target .featuredMatches section here
	)

	// this helps maintain correct order of UICV sections 
	let nextSection = snapshot.sectionIdentifiers.first(where: { ... })

	// convert CoreData snapshot into our snapshot
	snapshot = populateSnapshotWithFeaturedMatches(
		snapshot,
		with: cds,
		beforeSection: nextSection
	)

	// apply our updated snapshot
	gridSource.apply(snapshot, animatingDifferences: animated)
}

We also implemented some simple logic to maintain the order of sections. You can’t just appendSections each time you receive a set of data since your UI sections will constantly shuffle downwards.

func populateSnapshotWithFeaturedMatches(_ currentSnapshot: Snapshot, with cds: NSDiffableDataSourceSnapshot<String, NSManagedObjectID>, beforeSection: SectionItem? = nil) -> Snapshot {

	let eventMOIDs = cds.itemIdentifiers
	guard
		let sectionIdentifier = cds.sectionIdentifiers.first,
		eventMOIDs.count > 0
	else {
		return currentSnapshot
	}

	var snapshot = currentSnapshot

	let secitem = HomeDataSource.SectionItem.featuredMatches(events: eventMOIDs)
	if let nextSection = beforeSection {
		snapshot.insertSections([secitem], beforeSection: nextSection)
	} else {
		snapshot.appendSections([secitem])
	}

	// Here convert data from cds into 
	// GridItem instances added to secitem

	return snapshot
}

That’s…pretty much done. The rest of the work was carefully going over each FRC, check what its snapshot really is and build out the snapshot conversion into enums we are using.

Cell vending changed too - we take the NSManagedObjectID and ask the NSManagedObjectContext for the full object:

func object<T>(with moID: NSManagedObjectID) -> T? {
	do {
		let object = try moc.existingObject(with: moID)
		return object as? T
	} catch let err {
		return nil
	}
}

func cell(collectionView: UICollectionView, indexPath: IndexPath, item: GridItem) -> UICollectionViewCell {
	switch item {
		...
		case .sport(let moID):
			guard let sport: Sport = object(with: moID) else {
				preconditionFailure()
			}
			let cell: SportPickerCell = collectionView.dequeueReusableCell(forIndexPath: indexPath)
			cell.populate(with: sport.icon, caption: sport.name, isSelected: false)
			return cell
			
		...

As you can see, this is far from simple and easy. There’s lots of experimentation and verification to do. But the end result is a system that works really, really well.

Before compositional layout, an UI like this would be multiple nested collection views with flow layout — we had that and maintaining it was super hard and was never 100% crash-free. Now, having everything in one single UICollectionView is freaking amazing from performance aspect: everything scroll super smooth, memory consumption is very low thanks to Core Data efficiency and so far it looks to be crash-free 🤞🏻.

All of this is supported since iOS 13 so right now it’s pretty much guaranteed you can freely use it in just about any app.