Skip to content

Datasets Reference

philanthropy.datasets

Synthetic data generators and real reference datasets for donor analytics.

load_ciob_fundraising()

Load the NYC CIOB "Official Fundraising by City Agencies" registry.

A real open-government dataset: every not-for-profit that a New York City agency reported soliciting donations for, disclosed under the Conflicts of Interest Board's (CIOB) mandate. It is an affiliation registry, one row per (year, agency, nonprofit) link, not donor-level giving data.

It carries no gift amounts, donor records, or engagement labels, so it does not support the RFM / propensity modelling in the rest of this library; use :func:generate_synthetic_donor_data for that. It is included for honest, reproducible analysis of agency ↔ nonprofit fundraising relationships (e.g. most-solicited nonprofits, agency fundraising breadth).

Returns:

Type Description
pandas.DataFrame of shape (2336, 3)

Columns:

year : int Calendar year of the reported fundraising (2019–2024). agency : string NYC agency that solicited the donation. name_of_not_for_profit : string Beneficiary not-for-profit organisation.
Notes

Source: NYC Open Data, dataset basd-2jwn (Conflicts of Interest Board), https://data.cityofnewyork.us/City-Government/Official-Fundraising-by-City-Agencies/basd-2jwn. Distributed under the NYC Open Data Terms of Use (free public use). The CSV is vendored with the package, so this loader needs no network access.

Source code in philanthropy/datasets/_ciob.py 
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
def load_ciob_fundraising() -> pd.DataFrame:
    """Load the NYC CIOB "Official Fundraising by City Agencies" registry.

    A real open-government dataset: every not-for-profit that a New York City
    agency reported soliciting donations for, disclosed under the Conflicts of
    Interest Board's (CIOB) mandate. It is an **affiliation registry**, one row
    per ``(year, agency, nonprofit)`` link, **not** donor-level giving data.

    It carries no gift amounts, donor records, or engagement labels, so it does
    **not** support the RFM / propensity modelling in the rest of this library;
    use :func:`generate_synthetic_donor_data` for that. It is included for
    honest, reproducible analysis of agency ↔ nonprofit fundraising
    relationships (e.g. most-solicited nonprofits, agency fundraising breadth).

    Returns
    -------
    pandas.DataFrame of shape (2336, 3)
        Columns:

        ``year`` : int
            Calendar year of the reported fundraising (2019–2024).
        ``agency`` : string
            NYC agency that solicited the donation.
        ``name_of_not_for_profit`` : string
            Beneficiary not-for-profit organisation.

    Notes
    -----
    Source: NYC Open Data, dataset ``basd-2jwn`` (Conflicts of Interest Board),
    https://data.cityofnewyork.us/City-Government/Official-Fundraising-by-City-Agencies/basd-2jwn.
    Distributed under the NYC Open Data Terms of Use (free public use). The CSV
    is vendored with the package, so this loader needs no network access.
    """
    csv = files("philanthropy.datasets").joinpath(
        "data/ciob_official_fundraising.csv"
    )
    with csv.open("r", encoding="utf-8") as fh:
        return pd.read_csv(
            fh,
            dtype={
                "year": "int64",
                "agency": "string",
                "name_of_not_for_profit": "string",
            },
        )

generate_synthetic_donor_data(n_samples=1000, random_state=None)

Generate a realistic synthetic donor DataFrame for modelling and testing.

The returned dataset simulates a hospital's major-gifts prospect pool. Features are correlated in a domain-meaningful way:

  • Donors with more years_active and higher event_attendance_count have a monotonically increasing probability of being labelled as a major donor (is_major_donor = 1).
  • total_gift_amount is log-normally distributed and positively correlated with is_major_donor.
  • last_gift_date is sampled uniformly across the past five calendar years, with major donors skewed toward more recent activity.

Parameters:

Name Type Description Default
n_samples int

Number of synthetic donor records to generate.

 1000
random_state int or None

Seed for the NumPy random-number generator. Pass an integer to obtain a reproducible dataset; None draws a fresh seed on every call.

 None

Returns:

Name Type Description
df pd.DataFrame of shape (n_samples, 5)

A DataFrame with the following columns:

total_gift_amount : float Cumulative lifetime giving in USD. Drawn from a log-normal distribution (mu = 7.5, sigma = 1.4); major donors receive an additional multiplicative uplift of 3–8×. years_active : int Number of full calendar years since the donor's first recorded gift (range 1–30). Major-donor candidates are skewed toward longer tenure. last_gift_date : datetime Date of the most recent gift. Stored as datetime64[ns]. Major donors are skewed toward dates within the past two years. event_attendance_count : int Number of fundraising events attended (range 0–20). Higher values increase propensity-to-give probability. is_major_donor : int (0 or 1) Binary label indicating whether the donor is classified as a major gift prospect (gift capacity ≥ $25,000).

Examples:

>>> from philanthropy.datasets import generate_synthetic_donor_data
>>> df = generate_synthetic_donor_data(n_samples=500, random_state=42)
>>> df.shape
(500, 5)
>>> df.dtypes["is_major_donor"]
dtype('int64')
>>> bool(df["is_major_donor"].isin([0, 1]).all())
True
Notes

A latent giving capacity drives everything. It is a linear function of years_active, event_attendance_count and an unobserved wealth term, on a log-dollar scale. total_gift_amount is then drawn as a noisy realisation of that capacity, and is_major_donor as a soft threshold on it at :data:MAJOR_GIFT_CAPACITY. last_gift_date follows engagement.

The ordering matters. Capacity is a confounder that causes both the giving history and the label, so total_gift_amount is a legitimate predictor: informative, and limited by how well giving reveals capacity.

Before version 0.7.0 the label was drawn first and total_gift_amount was drawn conditional on the label. That inverted the domain's causal arrow, and it was measurable: a model given total_gift_amount beat the Bayes rate of the generator's own process by roughly 19 ROC-AUC points, which no model can legitimately do. Using cumulative lifetime giving to predict "is a major donor" is also the classic fundraising leakage this library exists to prevent, so the reference dataset was teaching the anti-pattern. last_gift_date was a second target-derived feature for the same reason.

The label remains statistically learnable and is not trivially predictable: held out on 4,000 rows, the documented feature set reaches ROC-AUC 0.814 and accuracy 0.759 against a Bayes accuracy ceiling of 0.806 given latent capacity. Sitting below that ceiling is the point.

The function never raises an error for valid inputs. Passing n_samples=0 returns an empty DataFrame with the correct column schema.

Source code in philanthropy/datasets/_generator.py 
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
def generate_synthetic_donor_data(
    n_samples: int = 1000,
    random_state: Optional[int] = None,
) -> pd.DataFrame:
    """Generate a realistic synthetic donor DataFrame for modelling and testing.

    The returned dataset simulates a hospital's major-gifts prospect pool.
    Features are correlated in a domain-meaningful way:

    * Donors with more ``years_active`` and higher ``event_attendance_count``
      have a monotonically increasing probability of being labelled as a
      major donor (``is_major_donor = 1``).
    * ``total_gift_amount`` is log-normally distributed and positively
      correlated with ``is_major_donor``.
    * ``last_gift_date`` is sampled uniformly across the past five calendar
      years, with major donors skewed toward more recent activity.

    Parameters
    ----------
    n_samples : int, default=1000
        Number of synthetic donor records to generate.
    random_state : int or None, default=None
        Seed for the NumPy random-number generator.  Pass an integer to
        obtain a reproducible dataset; ``None`` draws a fresh seed on every
        call.

    Returns
    -------
    df : pd.DataFrame of shape (n_samples, 5)
        A DataFrame with the following columns:

        ``total_gift_amount`` : float
            Cumulative lifetime giving in USD.  Drawn from a log-normal
            distribution (mu = 7.5, sigma = 1.4); major donors receive an
            additional multiplicative uplift of 3–8×.
        ``years_active`` : int
            Number of full calendar years since the donor's first recorded
            gift (range 1–30).  Major-donor candidates are skewed toward
            longer tenure.
        ``last_gift_date`` : datetime
            Date of the most recent gift.  Stored as ``datetime64[ns]``.
            Major donors are skewed toward dates within the past two years.
        ``event_attendance_count`` : int
            Number of fundraising events attended (range 0–20).  Higher
            values increase propensity-to-give probability.
        ``is_major_donor`` : int (0 or 1)
            Binary label indicating whether the donor is classified as a
            major gift prospect (gift capacity ≥ $25,000).

    Examples
    --------
    >>> from philanthropy.datasets import generate_synthetic_donor_data
    >>> df = generate_synthetic_donor_data(n_samples=500, random_state=42)
    >>> df.shape
    (500, 5)
    >>> df.dtypes["is_major_donor"]
    dtype('int64')
    >>> bool(df["is_major_donor"].isin([0, 1]).all())
    True

    Notes
    -----
    A latent **giving capacity** drives everything. It is a linear function of
    ``years_active``, ``event_attendance_count`` and an unobserved wealth term,
    on a log-dollar scale. ``total_gift_amount`` is then drawn as a noisy
    realisation of that capacity, and ``is_major_donor`` as a soft threshold on
    it at :data:`MAJOR_GIFT_CAPACITY`. ``last_gift_date`` follows engagement.

    The ordering matters. Capacity is a **confounder** that causes both the
    giving history and the label, so ``total_gift_amount`` is a legitimate
    predictor: informative, and limited by how well giving reveals capacity.

    Before version 0.7.0 the label was drawn first and ``total_gift_amount``
    was drawn *conditional on the label*. That inverted the domain's causal
    arrow, and it was measurable: a model given ``total_gift_amount`` beat the
    Bayes rate of the generator's own process by roughly 19 ROC-AUC points,
    which no model can legitimately do. Using cumulative lifetime giving to
    predict "is a major donor" is also the classic fundraising leakage this
    library exists to prevent, so the reference dataset was teaching the
    anti-pattern. ``last_gift_date`` was a second target-derived feature for
    the same reason.

    The label remains statistically learnable and is not trivially
    predictable: held out on 4,000 rows, the documented feature set reaches
    ROC-AUC 0.814 and accuracy 0.759 against a Bayes accuracy ceiling of 0.806
    given latent capacity. Sitting **below** that ceiling is the point.

    The function never raises an error for valid inputs.  Passing
    ``n_samples=0`` returns an empty DataFrame with the correct column
    schema.
    """
    rng = np.random.default_rng(random_state)

    # ------------------------------------------------------------------
    # Step 1: Generate structural features
    # ------------------------------------------------------------------
    years_active = rng.integers(1, 31, size=n_samples)          # 1–30 years
    event_attendance = rng.integers(0, 21, size=n_samples)       # 0–20 events

    # ------------------------------------------------------------------
    # Step 2: Latent giving capacity.
    #
    # This is the confounder, and the whole point of the ordering here. Capacity
    # causes BOTH the observed giving history and major-donor status. Nothing
    # below is drawn from the label, so no feature is a readout of the answer.
    #
    # Earlier versions drew the label first and then drew total_gift_amount
    # conditional on it, which inverted the domain's causal arrow: a model given
    # total_gift_amount could beat the Bayes rate of the generator's own process,
    # and using cumulative giving to predict "is a major donor" is the classic
    # fundraising leakage this library exists to prevent.
    # ------------------------------------------------------------------
    unobserved_wealth = rng.normal(0, 1, size=n_samples)
    log_capacity = (
        7.9                               # intercept, sets the base rate
        + 0.055 * years_active            # longer relationships run deeper
        + 0.075 * event_attendance        # engagement tracks affinity and means
        + 1.25 * unobserved_wealth        # wealth no column in this frame sees
    )

    # ------------------------------------------------------------------
    # Step 3: Observed giving FOLLOWS capacity.
    #
    # Donors realise some fraction of what they could give, imperfectly, so
    # total_gift_amount is a noisy proxy for capacity rather than a function of
    # the label. That makes it a legitimate predictor: informative, and bounded
    # by how well giving history reveals capacity.
    # ------------------------------------------------------------------
    total_gift_amount = np.round(
        rng.lognormal(mean=log_capacity - 1.6, sigma=0.6), 2
    )

    # ------------------------------------------------------------------
    # Step 4: The label ALSO follows capacity: a soft threshold at the
    # $25,000 gift capacity the docstring describes. Soft rather than hard
    # because a real prospect-research call is not a step function.
    # ------------------------------------------------------------------
    z = 1.6 * (log_capacity - np.log(MAJOR_GIFT_CAPACITY)) + 0.4 * rng.normal(
        0, 1, size=n_samples
    )
    propensity = 1.0 / (1.0 + np.exp(-z))
    is_major_donor = rng.binomial(1, propensity).astype(np.int64)

    # ------------------------------------------------------------------
    # Step 5: Recency follows ENGAGEMENT, not the label.
    #
    # This was also drawn from the label before (Beta for majors, uniform for
    # everyone else), which made last_gift_date a second target-derived feature.
    # More engaged donors have given more recently, which is the real mechanism.
    # ------------------------------------------------------------------
    reference_date = pd.Timestamp("2026-02-21")  # project snapshot date
    max_days = 365 * 5
    engagement = 1.0 / (1.0 + np.exp(-0.18 * (event_attendance - 8.0)))
    recency_days = np.zeros(n_samples, dtype=np.int64)
    if n_samples > 0:
        # Larger second Beta parameter skews toward 0, i.e. toward recent dates.
        recency_days = (
            rng.beta(1.0, 1.0 + 3.5 * engagement) * max_days
        ).astype(np.int64)

    last_gift_date = pd.to_datetime(
        reference_date - pd.to_timedelta(recency_days, unit="D")
    )

    # ------------------------------------------------------------------
    # Step 6: Assemble DataFrame
    # ------------------------------------------------------------------
    df = pd.DataFrame(
        {
            "total_gift_amount": total_gift_amount,
            "years_active": years_active.astype(np.int64),
            "last_gift_date": last_gift_date,
            "event_attendance_count": event_attendance.astype(np.int64),
            "is_major_donor": is_major_donor,
        }
    )

    return df

make_donor_dataset(n_donors=200, fiscal_year_start=7, start_year=2018, end_year=2024, lapse_rate=0.25, major_gift_threshold=10000.0, random_state=42)

Generate a synthetic, seeded gift-level donor dataset.

Parameters:

Name Type Description Default
n_donors int

Number of synthetic donors to generate.

 200
fiscal_year_start int

Reserved. Accepted for signature stability but not currently used when generating gift dates.

 7
start_year int

Earliest calendar year used when sampling gift dates.

 2018
end_year int

Latest calendar year used when sampling gift dates.

 2024
lapse_rate float

Reserved. Accepted for signature stability but not currently used when generating gift dates.

 0.25
major_gift_threshold float

Gift amount (inclusive) that marks a row as a major gift.

 10_000.0
random_state int or None

Seed for the NumPy random-number generator. Pass an integer to obtain a reproducible dataset; None draws a fresh seed every call.

 42

Returns:

Name Type Description
df DataFrame

A gift-level DataFrame, not a donor-level DataFrame: each donor contributes 1-5 gift rows, so len(df) > n_donors. Rows are sorted by gift_date and the index is reset. Columns:

donor_id : str Zero-padded donor id, e.g. D00001. gift_date : datetime64[ns] Sampled between start_year and end_year. gift_amount : float Log-normal gift amount, rounded to two decimal places. appeal_code : str One of ANNUAL, MAJOR, PLANNED, ONLINE, or EVENT. is_major_gift : bool True when gift_amount >= major_gift_threshold.

Examples:

>>> from philanthropy.datasets import make_donor_dataset
>>> df = make_donor_dataset(n_donors=5, random_state=0)
>>> df.shape
(15, 5)
>>> list(df.columns)
['donor_id', 'gift_date', 'gift_amount', 'appeal_code', 'is_major_gift']
>>> df['donor_id'].nunique()
5
Source code in philanthropy/datasets/_generator.py 
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
def make_donor_dataset(
    n_donors: int = 200,
    fiscal_year_start: int = 7,
    start_year: int = 2018,
    end_year: int = 2024,
    lapse_rate: float = 0.25,
    major_gift_threshold: float = 10_000.0,
    random_state: Optional[int] = 42,
) -> pd.DataFrame:
    """Generate a synthetic, seeded gift-level donor dataset.

    Parameters
    ----------
    n_donors : int, default=200
        Number of synthetic donors to generate.
    fiscal_year_start : int, default=7
        Reserved. Accepted for signature stability but not currently used
        when generating gift dates.
    start_year : int, default=2018
        Earliest calendar year used when sampling gift dates.
    end_year : int, default=2024
        Latest calendar year used when sampling gift dates.
    lapse_rate : float, default=0.25
        Reserved. Accepted for signature stability but not currently used
        when generating gift dates.
    major_gift_threshold : float, default=10_000.0
        Gift amount (inclusive) that marks a row as a major gift.
    random_state : int or None, default=42
        Seed for the NumPy random-number generator. Pass an integer to
        obtain a reproducible dataset; ``None`` draws a fresh seed every call.

    Returns
    -------
    df : pd.DataFrame
        A gift-level DataFrame, not a donor-level DataFrame: each donor
        contributes 1-5 gift rows, so ``len(df) > n_donors``. Rows are sorted
        by ``gift_date`` and the index is reset. Columns:

        ``donor_id`` : str
            Zero-padded donor id, e.g. ``D00001``.
        ``gift_date`` : datetime64[ns]
            Sampled between ``start_year`` and ``end_year``.
        ``gift_amount`` : float
            Log-normal gift amount, rounded to two decimal places.
        ``appeal_code`` : str
            One of ``ANNUAL``, ``MAJOR``, ``PLANNED``, ``ONLINE``, or ``EVENT``.
        ``is_major_gift`` : bool
            True when ``gift_amount >= major_gift_threshold``.

    Examples
    --------
    >>> from philanthropy.datasets import make_donor_dataset
    >>> df = make_donor_dataset(n_donors=5, random_state=0)
    >>> df.shape
    (15, 5)
    >>> list(df.columns)
    ['donor_id', 'gift_date', 'gift_amount', 'appeal_code', 'is_major_gift']
    >>> df['donor_id'].nunique()
    5
    """
    rng = np.random.default_rng(random_state)
    donor_ids = [f"D{str(i).zfill(5)}" for i in range(1, n_donors + 1)]
    records = []
    for donor_id in donor_ids:
        n_gifts = rng.integers(1, 6)
        for _ in range(n_gifts):
            year = rng.integers(start_year, end_year + 1)
            month = rng.integers(1, 13)
            day = rng.integers(1, 28)
            gift_date = pd.Timestamp(year=int(year), month=int(month), day=int(day))
            gift_amount = float(rng.lognormal(mean=5.5, sigma=1.2))
            gift_amount = round(gift_amount, 2)
            appeal_code = rng.choice(["ANNUAL", "MAJOR", "PLANNED", "ONLINE", "EVENT"])
            records.append(
                {
                    "donor_id": donor_id,
                    "gift_date": gift_date,
                    "gift_amount": gift_amount,
                    "appeal_code": appeal_code,
                }
            )
    df = pd.DataFrame(records).sort_values("gift_date").reset_index(drop=True)
    df["is_major_gift"] = df["gift_amount"] >= major_gift_threshold
    return df

fetch_kdd98_donors(*, data_home=None, download_if_missing=True)

Fetch the KDD Cup 1998 direct-mail donor learning set.

A real donor-level dataset: 95,412 individuals who gave at least once between June 1995 and June 1996, one row per donor, with a full per-promotion mail and response history (ADATE_2..ADATE_24, RDATE_2..RDATE_24, RAMNT_2..RAMNT_24) plus the outcome of the 1997 mailing being predicted (TARGET_B, TARGET_D). That date history is what makes as-of feature construction testable on it rather than only on synthetic data; see :func:generate_synthetic_donor_data for the synthetic equivalent used elsewhere in this library. Most date columns are encoded YYMM (year, month) rather than as a datetime dtype.

This is a read-only public research dataset, not your data. Nothing about your own donors, gifts, or environment is ever sent anywhere; the only network traffic this function makes is fetching the dataset file itself, once, to a local cache. It is never called automatically: no other function in this library imports it or calls it during fit or transform.

Under the dataset's terms of use, teaching or training material that uses it must not name the sponsoring organisation; cite it only as "KDD Cup 1998". This docstring follows that condition, and so should anything you write based on it.

Parameters:

Name Type Description Default
data_home str

Directory to cache the downloaded archive in. Defaults to the PHILANTHROPY_DATA environment variable if set, else ~/philanthropy_data.

 None
download_if_missing bool

If the archive is not already cached, download it. If False and the archive is not cached, raise OSError instead of reaching for the network.

 True

Returns:

Type Description
pandas.DataFrame of shape (95412, 481)

One row per donor, columns as documented in the data dictionary below. Column dtypes are pandas' own inference over the raw CSV; this function does not recode or impute any of them.

Raises:

Type Description
OSError

If the archive is not cached and download_if_missing is False, or if a downloaded archive fails its checksum check.
Notes

Source: the UCI KDD Archive, https://kdd.ics.uci.edu/databases/kddcup98/kddcup98.html. Field-by-field documentation: cup98dic.txt at the same location. Distributed for general research and educational use under the terms stated on that page, including the sponsor-naming restriction noted above and a request to notify the dataset's contacts of any published results.

The archive is not vendored with this package (its terms require an unmodified, individually-fetched copy); this function downloads it to a local cache on first use, the way sklearn.datasets.fetch_* functions do, and every later call reads the cached copy.

Source code in philanthropy/datasets/_kdd98.py 
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
def fetch_kdd98_donors(
    *, data_home: Optional[str] = None, download_if_missing: bool = True
) -> pd.DataFrame:
    """Fetch the KDD Cup 1998 direct-mail donor learning set.

    A real donor-level dataset: 95,412 individuals who gave at least once
    between June 1995 and June 1996, one row per donor, with a full
    per-promotion mail and response history (``ADATE_2``..``ADATE_24``,
    ``RDATE_2``..``RDATE_24``, ``RAMNT_2``..``RAMNT_24``) plus the outcome of
    the 1997 mailing being predicted (``TARGET_B``, ``TARGET_D``). That date
    history is what makes as-of feature construction testable on it rather
    than only on synthetic data; see :func:`generate_synthetic_donor_data` for
    the synthetic equivalent used elsewhere in this library. Most date columns
    are encoded ``YYMM`` (year, month) rather than as a `datetime` dtype.

    This is a **read-only public research dataset**, not your data. Nothing
    about your own donors, gifts, or environment is ever sent anywhere; the
    only network traffic this function makes is fetching the dataset file
    itself, once, to a local cache. It is never called automatically: no
    other function in this library imports it or calls it during `fit` or
    `transform`.

    Under the dataset's terms of use, teaching or training material that uses
    it must not name the sponsoring organisation; cite it only as "KDD Cup
    1998". This docstring follows that condition, and so should anything you
    write based on it.

    Parameters
    ----------
    data_home : str, default=None
        Directory to cache the downloaded archive in. Defaults to the
        ``PHILANTHROPY_DATA`` environment variable if set, else
        ``~/philanthropy_data``.

    download_if_missing : bool, default=True
        If the archive is not already cached, download it. If False and the
        archive is not cached, raise ``OSError`` instead of reaching for the
        network.

    Returns
    -------
    pandas.DataFrame of shape (95412, 481)
        One row per donor, columns as documented in the data dictionary
        below. Column dtypes are pandas' own inference over the raw CSV;
        this function does not recode or impute any of them.

    Raises
    ------
    OSError
        If the archive is not cached and `download_if_missing` is False, or
        if a downloaded archive fails its checksum check.

    Notes
    -----
    Source: the UCI KDD Archive,
    https://kdd.ics.uci.edu/databases/kddcup98/kddcup98.html. Field-by-field
    documentation: ``cup98dic.txt`` at the same location. Distributed for
    general research and educational use under the terms stated on that page,
    including the sponsor-naming restriction noted above and a request to
    notify the dataset's contacts of any published results.

    The archive is not vendored with this package (its terms require an
    unmodified, individually-fetched copy); this function downloads it to a
    local cache on first use, the way ``sklearn.datasets.fetch_*`` functions
    do, and every later call reads the cached copy.
    """
    cache_dir = _data_home(data_home)
    archive_path = os.path.join(cache_dir, "cup98lrn.zip")

    if not os.path.exists(archive_path):
        if not download_if_missing:
            raise OSError(
                f"{archive_path} is not cached and download_if_missing=False. "
                "Call with download_if_missing=True to fetch it."
            )
        _download(_LEARNING_URL, archive_path, _LEARNING_SHA256)

    with zipfile.ZipFile(archive_path) as archive:
        with archive.open(_LEARNING_MEMBER) as fh:
            return pd.read_csv(fh, low_memory=False)

make_donor_panel(n_donors=3000, n_years=7, start_fiscal_year=2018, include_encounters=False, random_state=None)

Generate a seeded multi-year donor panel with gift-level rows.

Unlike :func:generate_synthetic_donor_data, which returns one already-aggregated row per donor, this returns the raw tables a fundraising shop actually exports: a gift log, a donor table, and optionally a clinical-encounter table. Aggregating them is left to the caller, on purpose, because when you aggregate is the whole subject of :doc:the leakage tutorial </tutorials/avoiding_temporal_data_leakage>.

Fiscal years run 1 July to 30 June and are labelled by the calendar year in which they end, so fiscal year 2019 spans 2018-07-01 to 2019-06-30. A donor gives at most once per fiscal year.

Parameters:

Name Type Description Default
n_donors int

Number of donors in the panel. Every donor appears in donors; donors who never gave contribute no rows to gifts.

 3000
n_years int

Number of consecutive fiscal years. Deriving a "gave in the following year" label costs the last one, so the default yields six usable panel years.

 7
start_fiscal_year int

Label of the first fiscal year.

 2018
include_encounters bool

Also return an "encounters" table for the grateful-patient transformers. Off by default: most callers are not an academic medical center, and an unused encounter table invites the mistake of treating synthetic clinical rows as if they meant something.

 False
random_state int or None

Seed for the NumPy generator. Pass an integer for a reproducible panel; None draws a fresh seed on every call.

 None

Returns:

Name Type Description
panel dict of str to pandas.DataFrame

"gifts" One row per gift: donor_id (int), gift_date (datetime64[ns]), gift_amount (float), fiscal_year (int), appeal (str). Sorted by gift_date. Column names match what :class:~philanthropy.preprocessing.RFMTransformer requires. "donors" One row per donor: donor_id (int), first_gift_fy (float, NaN for donors who never gave), wealth_estimate (float, ~30% NaN by design, for :class:~philanthropy.preprocessing.WealthScreeningImputer), employer (str, sometimes empty). "encounters", only when include_encounters=True Zero or more rows per donor: donor_id (int), admit_date and discharge_date (datetime64[ns]), service_line (str).
Notes

There is no label column, deliberately. A label is a claim about a point in time, and shipping one pre-computed would hand every user the exact mistake this package exists to prevent. Derive it, as of the year you are scoring:

from philanthropy.datasets import make_donor_panel panel = make_donor_panel(n_donors=200, random_state=0) gifts = panel["gifts"] gave = set(zip(gifts["donor_id"], gifts["fiscal_year"])) label = int((7, 2020) in gave) # did donor 7 give in FY2020?

Every number here is invented. The panel establishes mechanisms and shapes, never magnitudes you should quote for your own program; for magnitudes measured on a real donor file, see :doc:/explanation/real_data_replication.

Examples:

>>> from philanthropy.datasets import make_donor_panel
>>> panel = make_donor_panel(n_donors=500, random_state=42)
>>> sorted(panel)
['donors', 'gifts']
>>> list(panel["gifts"].columns)
['donor_id', 'gift_date', 'gift_amount', 'fiscal_year', 'appeal']
>>> int(panel["gifts"]["fiscal_year"].min()), int(panel["gifts"]["fiscal_year"].max())
(2018, 2024)
>>> len(panel["donors"])
500

Straight into RFM features, which the cross-sectional generator cannot do:

>>> from philanthropy.preprocessing import RFMTransformer
>>> rfm = RFMTransformer().fit_transform(panel["gifts"])
>>> list(rfm.columns)
['donor_id', 'recency', 'frequency', 'monetary']

With encounters, for the grateful-patient path:

>>> panel = make_donor_panel(
...     n_donors=500, include_encounters=True, random_state=42
... )
>>> list(panel["encounters"].columns)
['donor_id', 'admit_date', 'discharge_date', 'service_line']
Source code in philanthropy/datasets/_panel.py 
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
def make_donor_panel(
    n_donors: int = 3000,
    n_years: int = 7,
    start_fiscal_year: int = 2018,
    include_encounters: bool = False,
    random_state: Optional[int] = None,
) -> Dict[str, pd.DataFrame]:
    """Generate a seeded multi-year donor panel with gift-level rows.

    Unlike :func:`generate_synthetic_donor_data`, which returns one
    already-aggregated row per donor, this returns the *raw* tables a
    fundraising shop actually exports: a gift log, a donor table, and
    optionally a clinical-encounter table. Aggregating them is left to the
    caller, on purpose, because *when* you aggregate is the whole subject of
    :doc:`the leakage tutorial </tutorials/avoiding_temporal_data_leakage>`.

    Fiscal years run 1 July to 30 June and are labelled by the calendar year in
    which they end, so fiscal year 2019 spans 2018-07-01 to 2019-06-30. A donor
    gives at most once per fiscal year.

    Parameters
    ----------
    n_donors : int, default=3000
        Number of donors in the panel. Every donor appears in ``donors``;
        donors who never gave contribute no rows to ``gifts``.
    n_years : int, default=7
        Number of consecutive fiscal years. Deriving a "gave in the following
        year" label costs the last one, so the default yields six usable panel
        years.
    start_fiscal_year : int, default=2018
        Label of the first fiscal year.
    include_encounters : bool, default=False
        Also return an ``"encounters"`` table for the grateful-patient
        transformers. Off by default: most callers are not an academic medical
        center, and an unused encounter table invites the mistake of treating
        synthetic clinical rows as if they meant something.
    random_state : int or None, default=None
        Seed for the NumPy generator. Pass an integer for a reproducible
        panel; ``None`` draws a fresh seed on every call.

    Returns
    -------
    panel : dict of str to pandas.DataFrame
        ``"gifts"``
            One row per gift: ``donor_id`` (int), ``gift_date``
            (datetime64[ns]), ``gift_amount`` (float), ``fiscal_year`` (int),
            ``appeal`` (str). Sorted by ``gift_date``. Column names match what
            :class:`~philanthropy.preprocessing.RFMTransformer` requires.
        ``"donors"``
            One row per donor: ``donor_id`` (int), ``first_gift_fy`` (float,
            NaN for donors who never gave), ``wealth_estimate`` (float, ~30%
            NaN by design, for
            :class:`~philanthropy.preprocessing.WealthScreeningImputer`),
            ``employer`` (str, sometimes empty).
        ``"encounters"``, only when ``include_encounters=True``
            Zero or more rows per donor: ``donor_id`` (int), ``admit_date``
            and ``discharge_date`` (datetime64[ns]), ``service_line`` (str).

    Notes
    -----
    **There is no label column, deliberately.** A label is a claim about a
    point in time, and shipping one pre-computed would hand every user the
    exact mistake this package exists to prevent. Derive it, as of the year you
    are scoring:

    >>> from philanthropy.datasets import make_donor_panel
    >>> panel = make_donor_panel(n_donors=200, random_state=0)
    >>> gifts = panel["gifts"]
    >>> gave = set(zip(gifts["donor_id"], gifts["fiscal_year"]))
    >>> label = int((7, 2020) in gave)   # did donor 7 give in FY2020?

    Every number here is invented. The panel establishes mechanisms and shapes,
    never magnitudes you should quote for your own program; for magnitudes
    measured on a real donor file, see
    :doc:`/explanation/real_data_replication`.

    Examples
    --------
    >>> from philanthropy.datasets import make_donor_panel
    >>> panel = make_donor_panel(n_donors=500, random_state=42)
    >>> sorted(panel)
    ['donors', 'gifts']
    >>> list(panel["gifts"].columns)
    ['donor_id', 'gift_date', 'gift_amount', 'fiscal_year', 'appeal']
    >>> int(panel["gifts"]["fiscal_year"].min()), int(panel["gifts"]["fiscal_year"].max())
    (2018, 2024)
    >>> len(panel["donors"])
    500

    Straight into RFM features, which the cross-sectional generator cannot do:

    >>> from philanthropy.preprocessing import RFMTransformer
    >>> rfm = RFMTransformer().fit_transform(panel["gifts"])
    >>> list(rfm.columns)
    ['donor_id', 'recency', 'frequency', 'monetary']

    With encounters, for the grateful-patient path:

    >>> panel = make_donor_panel(
    ...     n_donors=500, include_encounters=True, random_state=42
    ... )
    >>> list(panel["encounters"].columns)
    ['donor_id', 'admit_date', 'discharge_date', 'service_line']
    """
    if n_donors < 1:
        raise ValueError(f"n_donors must be at least 1; got {n_donors}.")
    if n_years < 2:
        raise ValueError(
            "n_years must be at least 2, because a 'gave in the following "
            f"year' label needs a following year; got {n_years}."
        )

    rng = np.random.default_rng(random_state)

    # --- The giving process. ------------------------------------------------
    # These draws, in this order, are the ones scripts/leakage_experiment.py
    # has always made. Anything added below must stay below, or the published
    # benchmark numbers stop reproducing.
    theta = rng.normal(0, 1.2, n_donors)
    drift = np.linspace(0.3, -0.3, n_years)
    gave = np.zeros((n_donors, n_years), dtype=bool)
    amount = np.zeros((n_donors, n_years))
    for j in range(n_years):
        p = 1.0 / (1.0 + np.exp(-(theta + drift[j])))
        gave[:, j] = rng.random(n_donors) < p
        amount[:, j] = np.where(
            gave[:, j], rng.lognormal(6 + 0.35 * theta, 0.8), 0.0
        )

    # --- Everything below is presentation, and draws after the process. -----
    fiscal_years = start_fiscal_year + np.arange(n_years)
    day_offset = rng.integers(0, 365, size=(n_donors, n_years))
    appeal_idx = rng.integers(0, len(_APPEALS), size=(n_donors, n_years))

    donor_ix, year_ix = np.nonzero(gave)
    fy = fiscal_years[year_ix]
    # Fiscal year N opens on 1 July of year N-1.
    fy_start = pd.to_datetime([f"{year - 1}-07-01" for year in fy])
    gift_date = fy_start + pd.to_timedelta(day_offset[donor_ix, year_ix], unit="D")

    gifts = pd.DataFrame(
        {
            "donor_id": donor_ix,
            "gift_date": gift_date,
            # Not rounded to cents, tempting as that is. scripts/leakage_
            # experiment.py aggregates these amounts, and rounding moved the
            # published min-max ranges by 0.001 AUC. A cosmetic decimal is not
            # worth invalidating a number that is already in the docs and the
            # paper. Round at the point of display instead.
            "gift_amount": amount[donor_ix, year_ix],
            "fiscal_year": fy,
            "appeal": [_APPEALS[i] for i in appeal_idx[donor_ix, year_ix]],
        }
    ).sort_values("gift_date", ignore_index=True)

    # NaN at a realistic rate, because a wealth screen that came back for every
    # record is not a wealth screen anyone has ever received.
    wealth = np.exp(11.0 + 0.6 * theta + rng.normal(0, 0.5, n_donors))
    wealth[rng.random(n_donors) < 0.30] = np.nan

    ever_gave = gave.any(axis=1)
    first_gift_fy = np.full(n_donors, np.nan)
    first_gift_fy[ever_gave] = fiscal_years[gave[ever_gave].argmax(axis=1)]

    donors = pd.DataFrame(
        {
            "donor_id": np.arange(n_donors),
            "first_gift_fy": first_gift_fy,
            "wealth_estimate": np.round(wealth, 2),
            "employer": rng.choice(_EMPLOYERS, size=n_donors),
        }
    )

    panel = {"gifts": gifts, "donors": donors}
    if include_encounters:
        panel["encounters"] = _encounters(rng, n_donors, fiscal_years)
    return panel