euromod.statistics ================== .. py:module:: euromod.statistics Below are listed the main public classes of the euromod.statistics module. .. list-table:: **Classes** :header-rows: 0 :widths: auto :class: summarytable * - :py:obj:`Statistics ` - Calculates statistics from EUROMOD simulation output. * - :py:obj:`StatisticsResult ` - Container for statistics calculation results, ready for Python consumption. * - :py:obj:`StatisticsTable ` - A single statistics table with Python-native data access. .. toctree:: :titlesonly: :maxdepth: 3 .. py:class:: Statistics(template_path: str, variable: str = None) Calculates statistics from EUROMOD simulation output. :param template_path: Path to the template XML file. :type template_path: str :param variable: Variable name for Variable-type templates. :type variable: str, optional :raises FileNotFoundError: If the template file does not exist. :raises ValueError: If the template XML cannot be parsed. .. rubric:: Overview .. list-table:: Methods :header-rows: 0 :widths: auto :class: summarytable * - :py:obj:`calculate `\ (baseline, reforms, pages, tables) - Calculate statistics from simulation output. * - :py:obj:`calculate_custom `\ (baseline, aggregate_stats, distributional_stats) - Calculate custom statistics. * - :py:obj:`list_pages `\ () - Return the list of page names in the template. * - :py:obj:`list_structure `\ () - Return the template structure as ``{page_name: [table_names]}``. * - :py:obj:`list_tables `\ (page) - Return table names in the template, or only those of ``page`` if given. .. rubric:: Methods .. py:method:: calculate(baseline, reforms=None, pages=None, tables=None) Calculate statistics from simulation output. :param baseline: Baseline simulation output. A DataFrame must contain the variables required by the template (including idperson, idhh, dwt and dag). Note: passing a DataFrame skips the keep_clr_data zero-copy fast path, so the data is converted to CLR arrays on each call. :type baseline: Simulation or pandas.DataFrame or polars.DataFrame :param reforms: Reform outputs for comparison templates. Entries may be Simulations or DataFrames and may be mixed with the baseline type. :type reforms: list of Simulation or pandas.DataFrame or polars.DataFrame, optional :param pages: If given, only these template pages are calculated (partial execution); the rest are skipped, which can be much faster. Use :meth:`list_pages` to discover names. Global prerequisite actions always run, but a kept page that depends on a skipped one may fail. :type pages: list[str], optional :param tables: If given, only these tables are calculated. Can be combined with `pages`. Use :meth:`list_tables` to discover names. :type tables: list[str], optional :returns: Calculation results with multiple access patterns. :rtype: StatisticsResult :raises ValueError: If required variables are missing from the simulation output, or if template type requires reforms but none are provided. :raises RuntimeError: If the EM_TemplateCalculator reports errors during preparation or calculation. .. py:method:: calculate_custom(baseline, aggregate_stats=None, distributional_stats=None) Calculate custom statistics. Accepts aggregate and distributional statistic definitions as Python dicts, maps them to C# ExternalStatisticAggregate and ExternalStatisticDistributional objects, builds a programmatic template, and runs the calculation using the baseline Simulation data. If this Statistics instance was initialized with a template, the custom statistics are combined with the template-driven calculations in a single pass. Otherwise, a minimal template is constructed for the custom statistics alone. :param baseline: Simulation output, or a DataFrame containing the income-list variables (and idperson, idhh, dwt, dag) referenced by the requested statistics. :type baseline: Simulation or pandas.DataFrame or polars.DataFrame :param aggregate_stats: Aggregate statistic definitions. Each dict should have: - name (str): statistic name - income_list (str): income list variable name - description (str, optional): description - source (str, optional): data source - year (str, optional): year for year values - amount (str, optional): amount value for the year - beneficiaries (str, optional): beneficiaries count for the year :type aggregate_stats: list[dict], optional :param distributional_stats: Distributional statistic definitions. Each dict should have: - name (str): statistic name - income_list (str): income list variable name - description (str, optional): description - measures (list[str], optional): distributional measures to calculate. Supported: "gini", "s80s20", "poverty_rate", "median", "mean", "atkinson", "mean_log_deviation", "population_count" :type distributional_stats: list[dict], optional :returns: Calculation results with multiple access patterns. :rtype: StatisticsResult :raises ValueError: If no custom statistics definitions are provided, or if definitions are invalid (missing required fields). :raises RuntimeError: If the EM_TemplateCalculator reports errors during preparation or calculation. .. py:method:: list_pages() -> list Return the list of page names in the template. .. py:method:: list_structure() -> dict Return the template structure as ``{page_name: [table_names]}``. Useful for discovering which pages/tables exist before requesting a subset via the ``pages``/``tables`` parameters of :meth:`calculate`. .. py:method:: list_tables(page: str = None) -> list Return table names in the template, or only those of ``page`` if given. .. py:class:: StatisticsResult(display_results=None, use_polars=False, errors=None, warnings=None) Container for statistics calculation results, ready for Python consumption. :param display_results: The C# DisplayResults object from EM_TemplateCalculator. :type display_results: DisplayResults (CLR object), optional :param use_polars: If True, DataFrames produced by this result will use polars (default: False). :type use_polars: bool, optional :param errors: Error messages from the ErrorCollector. :type errors: list[str], optional :param warnings: Warning messages (non-fatal) from the ErrorCollector. :type warnings: list[str], optional .. rubric:: Overview .. list-table:: Methods :header-rows: 0 :widths: auto :class: summarytable * - :py:obj:`to_dataframes `\ () - Convert each table in results to a DataFrame, keyed by table name. * - :py:obj:`to_dict `\ () - Full results as a nested ``dict`` matching the SP_ExecutableCaller JSON structure. * - :py:obj:`to_excel `\ (path) - Export results to Excel file via EM_Statistics ExportHandling. * - :py:obj:`to_json `\ (path) - Export results to JSON file in SP_ExecutableCaller-compatible format. .. rubric:: Methods .. py:method:: to_dataframes() -> dict Convert each table in results to a DataFrame, keyed by table name. :returns: Dictionary mapping table name to its DataFrame representation. Uses the same DataFrame library (pandas or polars) as the input Simulation. :rtype: dict[str, pandas.DataFrame | polars.DataFrame] .. py:method:: to_dict() -> dict Full results as a nested ``dict`` matching the SP_ExecutableCaller JSON structure. The returned dictionary has the following shape:: { "info": {"title": str, "subtitle": str, "button": str, "description": str}, "pages": [ { "name": str, "title": str, "subtitle": str, "description": str, "tables": [ { "name": str, "title": str, "subtitle": str, "columns": [{"name": str, "title": str}], "rows": [{"name": str, "title": str}], "cells": [[{"displayValue": str, "value": float, "isStringValue": bool}]] } ] } ], "prepared": bool, "calculated": bool } :returns: Nested dictionary matching the SP_ExecutableCaller JSON schema. :rtype: dict .. py:method:: to_excel(path: str) -> None Export results to Excel file via EM_Statistics ExportHandling. Uses the C# ExportHandling.ExportSinglePackage method to generate an Excel workbook from the DisplayResults object, then writes the resulting MemoryStream to the specified file path. :param path: File path where the Excel (.xlsx) output will be written. :type path: str :raises ValueError: If no DisplayResults are available (calculation did not complete). :raises RuntimeError: If the C# ExportHandling reports an error during export. :raises OSError: If the file cannot be written to the specified path. .. py:method:: to_json(path: str) -> None Export results to JSON file in SP_ExecutableCaller-compatible format. Serializes the results using `to_dict()` and writes to the specified file. Applies sentinel value substitution for special float values: - NaN → 99999998 - +Infinity → 99999999 - -Infinity → -99999999 :param path: File path where the JSON output will be written. :type path: str :raises OSError: If the file cannot be written to the specified path. .. py:class:: StatisticsTable A single statistics table with Python-native data access.