euromod.eu_euromod.calibrate.data_preparation ============================================= .. py:module:: euromod.eu_euromod.calibrate.data_preparation .. autoapi-nested-parse:: From a EUROMOD run to the household sample the tax-benefit function is fitted on. Two steps, each a function: :func:`simulate` Runs one system of the EU model on its input dataset and returns EUROMOD's individual-level output. :func:`household_sample` Turns that output into one row per household: 1. The *head* of a household is its main earner: the member with the highest earnings (``ils_earns``), the oldest one if earnings are equal. If age is equal too, the member listed first in the output. 2. Households are kept when the head's age is within ``head_age`` and, by default, when no member has self-employment income (``yse``). The self-employed face different rules and their incomes are measured less reliably. 3. Incomes, taxes and benefits are summed over the members of a household. 4. Households with non-positive gross income, original income, disposable income or income after taxes are dropped. Households whose taxes (``ils_taxsim``) are negative in total are kept: refundable tax credits are support like a benefit, and dropping their recipients would make the sample depend on whether a country pays its support through the tax system. ``drop_negative_tax=True`` drops them, as version 0.4 did. 5. The bottom and top of the gross-income distribution are trimmed at weighted percentiles (``trim``). 6. Incomes are divided by the weighted mean gross income of the households that remain, so that 1.0 is mean income. The mean is taken over *all* remaining households: subgroups such as married households are selected afterwards and share the same scale, which keeps their estimates comparable. :func:`prepare_sample` does both in one call. What this module needs from the model ------------------------------------- Variables ``idhh, dag, dms, dwt, yse, idmother, idfather`` and income lists ``ils_earns, ils_origy, ils_dispy, ils_taxsim, ils_benmt, ils_bennt``, plus the income list chosen as gross income (default ``ils_origrepy``). A married head is ``dms == 2``. A child is a member aged 18 or under with a parent in the data. A system name starts with its two-letter country code (``PT_2022``). Paths are always arguments. Nothing is read from the environment. Below are listed the main public classes of the euromod.eu_euromod.calibrate.data_preparation module. .. list-table:: **Function** :header-rows: 0 :widths: auto :class: summarytable * - :py:obj:`household_sample `\ (output, \*, income, disposable, head\_age, trim, drop\_self\_employed, drop\_negative\_tax) - Build the household sample from EUROMOD's individual-level output. * - :py:obj:`prepare_sample `\ (model\_path, data\_path, system, \*, dataset, constants, addons, switches, cache\_dir, \*\*sample\_options) - Run a system and build its household sample: :func:`simulate` followed by * - :py:obj:`simulate `\ (model\_path, data\_path, system, \*, dataset, constants, addons, switches, cache\_dir) - Run one system of the EU model and return EUROMOD's individual-level output. Functions --------- .. py:function:: household_sample(output, *, income='ils_origrepy', disposable='ils_dispy', head_age=(25, 60), trim=(2.5, 99), drop_self_employed=True, drop_negative_tax=False) Build the household sample from EUROMOD's individual-level output. The steps are described at the top of this module. :param output: Individual-level output of a EUROMOD run, e.g. from :func:`simulate`. :type output: pandas.DataFrame :param income: Income list used as gross income. Default is original income plus replacement income, ``ils_origrepy``. :type income: :obj:`str` :param disposable: ``"ils_dispy"``, or ``"ils_dispy_sicee"`` for disposable income before employee social insurance contributions (``ils_dispy + ils_sicee``). :type disposable: :obj:`str` :param head_age: Youngest and oldest age of the head, both included. :type head_age: (int, int) :param trim: Weighted percentiles of household gross income below and above which households are dropped. :type trim: (float, float) :param drop_self_employed: Drop households in which any member has self-employment income. :type drop_self_employed: :obj:`bool` :param drop_negative_tax: Drop households whose taxes (``ils_taxsim``, summed over the members) are negative. Off by default; version 0.4 always dropped them. :type drop_negative_tax: :obj:`bool` :returns: One row per household, sorted by ``idhh``, with - ``dwt``, ``hh_size``, ``head_age``, ``dms``, ``married``, ``n_children``; - the household totals of the income lists (``ils_origrepy``, ``ils_dispy``, ``ils_benmt``, ..., and ``ils_ben`` and ``ils_sicee`` when the output has them) and ``posttax_income`` (gross income minus ``ils_taxsim``); - ``income_norm`` and ``disposable_norm``: gross and disposable income divided by mean gross income; - ``net_tax_rate = 1 - disposable / income`` (taxes net of benefits) and ``tax_rate = 1 - posttax_income / income`` (taxes only). ``sample.attrs["mean_income"]`` holds the mean used for normalising. :rtype: pandas.DataFrame .. rubric:: Example >>> sample = household_sample(simulate(MODEL, DATA, "PT_2022")) >>> married = sample.query("married") .. py:function:: prepare_sample(model_path, data_path, system, *, dataset=None, constants=None, addons=(), switches=(), cache_dir=None, **sample_options) Run a system and build its household sample: :func:`simulate` followed by :func:`household_sample`. :param model_path: As in :func:`simulate`. :param data_path: As in :func:`simulate`. :param system: As in :func:`simulate`. :param dataset: As in :func:`simulate`. :param constants: As in :func:`simulate`. :param addons: As in :func:`simulate`. :param switches: As in :func:`simulate`. :param cache_dir: As in :func:`simulate`. :param \*\*sample_options: ``income``, ``disposable``, ``head_age``, ``trim``, ``drop_self_employed``, ``drop_negative_tax``, as in :func:`household_sample`. :returns: One row per household. :rtype: pandas.DataFrame .. rubric:: Example >>> baseline = prepare_sample(MODEL, DATA, "PT_2022") >>> reform = prepare_sample(MODEL, DATA, "PT_2022_ref") .. py:function:: simulate(model_path, data_path, system, *, dataset=None, constants=None, addons=(), switches=(), cache_dir=None) Run one system of the EU model and return EUROMOD's individual-level output. :param model_path: Folder of the EUROMOD model. :type model_path: :obj:`str` :param data_path: Folder with the input datasets (``.txt``, directly in the folder or in one subfolder per country). :type data_path: :obj:`str` :param system: Name of the system, e.g. ``"PT_2022"``, or of a counterfactual system defined in the model, e.g. ``"PT_2022_ref"``. :type system: :obj:`str` :param dataset: Name of the input dataset. Default is the system's best-match dataset. A baseline and a counterfactual are only comparable on the same dataset, so pass it when the counterfactual system has no best match of its own. :type dataset: :obj:`str`, optional :param constants: Model constants to override for this run, e.g. ``{"$tin00_rate9": 0.58}``. For a constant defined per group, use the pair as key: ``{("$f_cpi", "2023"): 1.02}``. :type constants: :obj:`dict`, optional :param addons: Passed to :meth:`euromod.System.run`. :type addons: list, optional :param switches: Passed to :meth:`euromod.System.run`. :type switches: list, optional :param cache_dir: Folder in which to keep the output and reuse it on the next call. Use one folder per scenario: beside each output a small ``.json`` records the dataset, constants, add-ons and switches that produced it, and asking for the same system with different ones is an error. :type cache_dir: :obj:`str`, optional :returns: One row per individual. :rtype: pandas.DataFrame .. rubric:: Example >>> output = simulate(MODEL, DATA, "PT_2022") >>> reform = simulate(MODEL, DATA, "PT_2022", constants={"$tin00_rate9": 0.58})