How to Configure Zingg to Automatically Treat Exact_Match Columns as Duplicates Regardless of Other Data

·May 01, 2024 02:38 AM

when I say "exact_match" how do I make it so zingg just knows that if those match it will always be an duplicate or automatically match them no matter what the other columns say?

20 comments

· Sorted by Oldest

Sonal G.
·
If you label enough records with those characteristics, Zingg will learn. Automatic deterministic matching is not supported in the community version. If you prefer, you could preprocess such records and match them through standard sql joins
Brandon M.
·
ok and is there a way to take two disparate data sets and instead of finding duplicates, merge them?
Brandon M.
·
i could combine them into one csv but it'd be nice to just have two csvs and 2 sources
Sonal G.
·
The data pipe in Zingg is an array, so is the data attribute in the config. You can send in as many as you want. The match will run on all the datasets together. Have I got your question right?
Brandon M.
·
if i have places.csv and another_set_of_places.csv
Brandon M.
·
how do i tell zing to find duplicates that are in both places.csv and another_set_of_places.csv?
Sonal G.
·
Got it. Just add as many data sources with the same schema in the config or through python and run match
Brandon M.
·
whenever I do findAndLAbel it just pulls from the same source
Brandon M.
·
it will just compare two rows from places.csv to eachother
Brandon M.
·
never one from places.csv and one from another_set_of_places.csv

Brandon M.

 "data": [
    {
      "name": "places",
      "format": "csv",
      "props": {
        "location": "places.csv",
        "delimiter": ",",
        "header": true
      },
      "schema": "pursuit_id string,state_abbr string,ori_id string,display_name string,url string,CaseSafe_ID string"
    },
    {
      "name": "fs_places",
      "format": "csv",
      "props": {
        "location": "fs_places.csv",
        "delimiter": ",",
        "header": true
      },
      "schema": "pursuit_id string,state_abbr string,ori_id string,display_name string,url string,CaseSafe_ID string"
    }
  ],

Sonal G.
·
How many rounds have you run? Do you have any duplicates within the first source?
Sonal G.
·
What happens when you reverse the order of data in the config?
Brandon M.
·
no each source won't have any duplicates in the same csv file
👍1
Brandon M.
·
and I've asked this 1000 times, so, I apologize but I always end up with the output csv's being in different order. The schema in the data array must match the csv headers, and, in the fieldDefinition array it doesn't matter the order?

Sonal G.
·
If you label enough records with those characteristics, Zingg will learn. Automatic deterministic matching is not supported in the community version. If you prefer, you could preprocess such records and match them through standard sql joins
Brandon M.
·
ok and is there a way to take two disparate data sets and instead of finding duplicates, merge them?
Brandon M.
·
i could combine them into one csv but it'd be nice to just have two csvs and 2 sources
Sonal G.
·
The data pipe in Zingg is an array, so is the data attribute in the config. You can send in as many as you want. The match will run on all the datasets together. Have I got your question right?
Brandon M.
·
if i have places.csv and another_set_of_places.csv
Brandon M.
·
how do i tell zing to find duplicates that are in both places.csv and another_set_of_places.csv?
Sonal G.
·
Got it. Just add as many data sources with the same schema in the config or through python and run match
Brandon M.
·
whenever I do findAndLAbel it just pulls from the same source
Brandon M.
·
it will just compare two rows from places.csv to eachother
Brandon M.
·
never one from places.csv and one from another_set_of_places.csv

Brandon M.

 "data": [
    {
      "name": "places",
      "format": "csv",
      "props": {
        "location": "places.csv",
        "delimiter": ",",
        "header": true
      },
      "schema": "pursuit_id string,state_abbr string,ori_id string,display_name string,url string,CaseSafe_ID string"
    },
    {
      "name": "fs_places",
      "format": "csv",
      "props": {
        "location": "fs_places.csv",
        "delimiter": ",",
        "header": true
      },
      "schema": "pursuit_id string,state_abbr string,ori_id string,display_name string,url string,CaseSafe_ID string"
    }
  ],

Sonal G.
·
How many rounds have you run? Do you have any duplicates within the first source?
Sonal G.
·
What happens when you reverse the order of data in the config?
Brandon M.
·
no each source won't have any duplicates in the same csv file
👍1
Brandon M.
·
and I've asked this 1000 times, so, I apologize but I always end up with the output csv's being in different order. The schema in the data array must match the csv headers, and, in the fieldDefinition array it doesn't matter the order?