How do you detect duplicate towing job records?

Establish what the records mean

Record the source system, export period, record type and identifier. Check whether a later export contains updated versions of earlier rows. Define which fields establish identity and which describe a change. Preserve an untouched source copy and keep a correction log.

A worked example

Fictional example: Export A contains J1 booked and Export B contains J1 completed. These are versions of one job, not two completed jobs. Two separate rows J2 and J3 each collect $180 but represent different requests; deduplicating by amount would delete valid work. Two payment IDs linked to J2 may be split payments rather than duplicate receipts.

Choose the next action

Create a rule for exact repeated rows and a separate review path for conflicting versions. Do not silently pick the newest-looking row if timestamps or source semantics are unclear. The lab rejects duplicate IDs within a record type so you must resolve the conflict before calculating a clean summary. Keep payments and jobs in their own identity spaces.

Check whether it worked

Reconcile row counts and totals before and after each approved correction. List removed exact duplicates and retained updated versions with their evidence. Review a sample for accidental merges. If ambiguity remains, keep it unresolved rather than presenting a falsely precise job or collection total.

A repeatable review order

1. Source export and period. Record the evidence and mark any missing answer as unknown.
2. Record type and stable ID. Record the evidence and mark any missing answer as unknown.
3. Exact duplicate versus changed version. Record the evidence and mark any missing answer as unknown.
4. Conflict resolution evidence. Record the evidence and mark any missing answer as unknown.
5. Before/after count and amount checks. Record the evidence and mark any missing answer as unknown.