Mark custom object record as duplicate via Apex for large (100,000+) data sets
Problem statement Let's say I have a data load that inserts 100,000 unique records of a custom object - call it MyObject__c. Each record has a lookup to a Contact record that is populated. I want to iterate over the entire list of 100,000 inserted records and find all of the records of MyObject that look up to the same Contact, and then set a checkbox on all but one record that indicates they are 'duplicates'. Example: there are 5 record of MyObject (out of 100,000) that all look up to the Contact MyContact (SFID 123456789). After processing the entire list of 100,000 records, I'd have four of the five records have a 'duplicate reference' checkbox marked, with the fifth one having it unchecked. Basically a uniqueness check on the lookup. What would be the best way to accomplish something like this? My current thought process is to build an Apex batch job; The batch runs the query to get all MyObject records that need to be processed (100,000) and splits them in...