ny_brfss_health_indicators

[goutamabrol]

Adds the NewYork_BRFSS_Health_Indicators import under statvar_imports/us_newyork/ny_brfss_health_indicators/ for New York State Department of Health county-level BRFSS data.

Data Source: New York State Department of Health (NYSDOH) Behavioral Risk Factor Surveillance System (BRFSS), covering all 62 NY counties and NYC across 5 survey rounds (2014, 2016, 2018, 2021, 2024).
Coverage: Maps 75 StatVars (20 canonical + 55 provisional) across chronic diseases, health behaviors, and healthcare utilization, generating 13,479 observations from 17,700 raw records (ignoring 3,081 non-county regional rows and 1,140 suppressed values).
Downloader (download.py): Directly downloads full raw unpivoted dataset (17,700 records) from Socrata API with session pooling, exponential backoff retries, request logging, and atomic file replacement.
Mappings: Long-format PV map (ny_brfss_health_indicators_pv_map.csv) mapping columns (years, region_county, unadjusted_rate), all 62 counties + NYC to Census FIPS DCIDs (geoId/36...), #IgnoreRow for non-county regions.
Automation (manifest.json): Configured for Cloud Batch execution with download.py, stat_var_processor.py (--existing_statvar_mcf, --output_counters, node_mcf), and annual cron.
Validation (validation_config.json): Enforces historical deletion threshold of 0.1% (DELETED_RECORDS_PERCENT) and date freshness allowing up to 3-year publication lag for triennial survey releases (SQL_VALIDATOR).

Cloud Batch Verified: End-to-end job completed with status SUCCEEDED.

PR Checklist: link

Cloud Batch Job (test) - link

Validation: link

CRA Report - link

[gemini-code-assist]

Code Review

This pull request introduces a new data import for the New York State eBRFSS health indicators, including a downloader script, configuration files, and test data. The feedback highlights several issues in download.py, such as unused imports, an unused mapping dictionary, and an unused helper function, as well as unreachable code caused by logging.fatal. Additionally, the test dataset sample_input.csv is pivoted and does not match the unpivoted production format, which prevents the tests from properly validating the property-value mapping.