diff --git a/docs/about/about.rst b/docs/about/about.rst index 1c59d9421..829e5a12a 100644 --- a/docs/about/about.rst +++ b/docs/about/about.rst @@ -10,7 +10,7 @@ Data scientists spend a huge amount of time on data pre-processing and transform It would be great (we thought back in the day) to gather the most frequently used data pre-processing techniques and transformations in a library, from which we could pick and choose the transformation that we need, and use it just like we would use any other -sklearn class. This was the original vision for Feature-engine. +sklearn class. This was the original vision for feature-engine. Feature-engine is an open source Python package originally designed to support the online course `Feature Engineering for Machine Learning `_, @@ -21,13 +21,13 @@ growing international community is beginning to lead the development. Governance ---------- -The decision making process and governance structure of Feature-engine is laid out in +The decision making process and governance structure of feature-engine is laid out in the :ref:`**governance document** `. Core contributors ----------------- -The following people are currently core contributors to Feature-engine’s development +The following people are currently core contributors to feature-engine’s development and maintenance: .. include:: authors.rst @@ -36,7 +36,7 @@ and maintenance: Former core contributors ------------------------ -The following people are former core contributors to Feature-engine’s development +The following people are former core contributors to feature-engine’s development and maintenance: .. include:: former_authors.rst @@ -49,11 +49,11 @@ Contributors | -A growing international community is beginning to lead Feature-engine's development. -You can learn more about Feature-engine's Contributors in the +A growing international community is beginning to lead feature-engine's development. +You can learn more about feature-engine's Contributors in the `GitHub contributors page `_. -Citing Feature-engine +Citing feature-engine --------------------- .. image:: https://zenodo.org/badge/163630824.svg @@ -64,8 +64,8 @@ Citing Feature-engine | -If you use Feature-engine in a scientific publication, you can cite the following paper: -Galli, S., (2021). `Feature-engine: A Python package for feature engineering for machine learning. `_ +If you use feature-engine in a scientific publication, you can cite the following paper: +Galli, S., (2021). `feature-engine: A Python package for feature engineering for machine learning. `_ Journal of Open Source Software, 6(65), 3642. Bibtex entry: @@ -81,13 +81,13 @@ Bibtex entry: number = {65}, pages = {3642}, author = {Soledad Galli}, - title = {Feature-engine: A Python package for feature engineering for machine learning}, + title = {feature-engine: A Python package for feature engineering for machine learning}, journal = {Journal of Open Source Software} } -You can also find a DOI (digital object identifier) for every version of Feature-engine +You can also find a DOI (digital object identifier) for every version of feature-engine on `zenodo.org `_; use the BibTeX on this site to reference specific versions of the software. diff --git a/docs/about/governance.rst b/docs/about/governance.rst index 4282ae116..d6b39b2e8 100644 --- a/docs/about/governance.rst +++ b/docs/about/governance.rst @@ -4,7 +4,7 @@ Governance ========== The purpose of this document is to formalize the governance process used by the -Feature-engine project and clarify how decisions are made and how the community works +feature-engine project and clarify how decisions are made and how the community works together. This is the first version of our governance policy and will be updated as our community grows and more of us take on different roles. @@ -26,7 +26,7 @@ Core Contributors are community members who are dedicated to the continued devel of the project through ongoing engagement with the community. Core Contributors are expected to review code contributions, can approve and merge pull requests, can decide on the fate of pull requests, and can be involved in deciding major changes to the -Feature-engine API. Core Contributors determine who can join as a Core Contributor. +feature-engine API. Core Contributors determine who can join as a Core Contributor. Founder and Leadership @@ -38,7 +38,7 @@ dissemination of the project. In the tradition of Python, Sole is referred to as “benevolent dictator for life” (BDFL) of the project, or simply, “the founder”. From a governance perspective, the BDFL has a special role in that they provide vision, thought leadership, and high-level direction for the project’s members and contributors. -The BDFL has the authority to make all final decisions for the Feature-engine Project. +The BDFL has the authority to make all final decisions for the feature-engine Project. However, in practice, the BDFL, chooses to defer that authority to the consensus of the community discussion channels and the Core Contributors. The BDFL can also propose and vote for new Core Contributors. diff --git a/docs/about/index.rst b/docs/about/index.rst index f90f7e3e2..99b6ecc9c 100644 --- a/docs/about/index.rst +++ b/docs/about/index.rst @@ -4,9 +4,9 @@ About ===== -In this section you will find information about the Feature-engine's origin, main +In this section you will find information about the feature-engine's origin, main developers, roadmap and overall vision for the package. You will also find information -about how to cite Feature-engine and our main sponsors. +about how to cite feature-engine and our main sponsors. .. toctree:: :maxdepth: 1 diff --git a/docs/about/roadmap.rst b/docs/about/roadmap.rst index 80666cb79..be172424a 100644 --- a/docs/about/roadmap.rst +++ b/docs/about/roadmap.rst @@ -2,7 +2,7 @@ Roadmap ======= This document provides general directions on what the core contributors would like to -see developed in Feature-engine. As resources are limited, we can't promise when or if +see developed in feature-engine. As resources are limited, we can't promise when or if the transformers listed here will be included in the library. We welcome all the help we can get to support this vision. If you are interested in contributing, please get in touch. @@ -18,7 +18,7 @@ Feature-engine makes data engineering easy by allowing the selection of feature directly within its transformers. It also interlaces well with exploratory data analysis (EDA) by returning dataframes for easy data exploration. -Feature-engine’s transformers preserve Scikit-learn functionality with the methods fit() +Feature-engine’s transformers preserve scikit-learn functionality with the methods fit() and transform() and can be integrated into a Pipeline to simplify putting the model in production. @@ -32,14 +32,14 @@ unwanted stages of the development. Vision ------ -At the moment, Feature-engine's functionality is tailored to tabular data, with numerical, +At the moment, feature-engine's functionality is tailored to tabular data, with numerical, categorical, or datetime variables. We started supporting the creation of features for time series forecasting in 2022. -But we would like to extend Feature-engine's functionality to work with text and time +But we would like to extend feature-engine's functionality to work with text and time series, as well as, expand its current functionality for tabular data. -In the following figure we show the overall structure and vision for Feature-engine: +In the following figure we show the overall structure and vision for feature-engine: .. figure:: ../images/FeatureEnginePackageStructure.png :align: center @@ -59,9 +59,9 @@ in the package: Transformers for tabular data -The current transformations supported by Feature-engine return features that are easy +The current transformations supported by feature-engine return features that are easy to interpret, and the effects of the transformations are clear and easy to understand. -The original aim of Feature-engine was to provide technology that is suitable to create +The original aim of feature-engine was to provide technology that is suitable to create models that will be used in real settings, and return understandable variables. Having said this, more and more, users are requesting features to combine or transform @@ -99,7 +99,7 @@ Goals Our main goals are: - Continue maintaining a high-quality, well-documented collection of canonical tools for data processing. -- Expand the documentation with more examples about Feature-engine's functionality. +- Expand the documentation with more examples about feature-engine's functionality. - Expand the documentation with more detail on how to contribute to the package. - Expand the library's functionality as per the precedent paragraphs. diff --git a/docs/contribute/code_of_conduct.rst b/docs/contribute/code_of_conduct.rst index 15ff718bf..6ea336b3d 100644 --- a/docs/contribute/code_of_conduct.rst +++ b/docs/contribute/code_of_conduct.rst @@ -3,12 +3,12 @@ Code of Conduct Feature-engine is an open source Python project. We follow the `Python Software Foundation Code of Conduct `_. -All interactions among members of the Feature-engine community must meet those +All interactions among members of the feature-engine community must meet those guidelines. This includes (but is not limited to) interactions through the mailing list, GitHub and StackOverflow. Everyone is expected to be open, considerate, and respectful of others no matter what their position is within the project. We show gratitude for any contribution, big or -small. We welcome feedback and participation. We want to make Feature-engine a nice, +small. We welcome feedback and participation. We want to make feature-engine a nice, welcoming and safe place for you to do your first contribution to open source, and why not the second, the third and so on :). diff --git a/docs/contribute/contribute_code.rst b/docs/contribute/contribute_code.rst index 2b1c51283..4e85515ec 100644 --- a/docs/contribute/contribute_code.rst +++ b/docs/contribute/contribute_code.rst @@ -5,10 +5,10 @@ Contribute Code =============== -Contributing code to Feature-engine is fun and easy. If you want to make a code contribution, +Contributing code to feature-engine is fun and easy. If you want to make a code contribution, you can check the `issue tracker `_ for already requested and wanted functionality. Alternatively, you can create a new issue -with functionality you would like to see included in Feature-engine and then work it through. +with functionality you would like to see included in feature-engine and then work it through. Contributing workflow @@ -28,7 +28,7 @@ A typical contributing workflow goes like this: 10. **Test** the code. 11. **Review** the code with us. 12. Make the **changes** and commit them to your fork, using the same branch created in 5. -13. When it is ready, we will **merge** your contribution into Feature-engine's source code base. +13. When it is ready, we will **merge** your contribution into feature-engine's source code base. To avoid extra work or duplication, it is important that we communicate right from the beginning, so we can build together a clear understanding of how you would like to get involved @@ -39,10 +39,10 @@ In the rest of the document, we will describe steps 3 to 13 in more detail. Fork the Repository ------------------- -When you fork the repository, you create a copy of Feature-engine's source code into -your Github account, which you can edit. To fork Feature-engine's repository, click the +When you fork the repository, you create a copy of feature-engine's source code into +your Github account, which you can edit. To fork feature-engine's repository, click the **fork** button in the upper right corner of -`Feature-engine's Repository `_. +`feature-engine's Repository `_. .. figure:: ../images/fork.png :figclass: align-center @@ -57,7 +57,7 @@ Once you forked the repository, clone the fork to your local machine. $ git clone https://github.com//feature_engine -2. Set up an ``upstream`` remote from where you can pull the latest code changes occurring in the main Feature-engine repository:: +2. Set up an ``upstream`` remote from where you can pull the latest code changes occurring in the main feature-engine repository:: $ git remote add upstream https://github.com/feature-engine/feature_engine.git @@ -72,7 +72,7 @@ You should see both your fork (origin) and the main repository (upstream) linked upstream https://github.com/feature-engine/feature_engine.git (fetch) upstream https://github.com/feature-engine/feature_engine.git (push) -Keep in mind that Feature-engine is being actively developed, so you may need to update +Keep in mind that feature-engine is being actively developed, so you may need to update your fork regularly. See below how to **Keep your fork up to date**. Set up the Development Environment @@ -81,13 +81,13 @@ Set up the Development Environment After creating a local copy of the repo, you need to set up the development environment. Setting up a development environment will ensure that you have all the libraries you need for the development, no more and no less. These libraries include -`Feature-engine dependencies `_, -like Pandas, NumPy and Scikit-learn and also +`feature-engine dependencies `_, +like pandas, NumPy and scikit-learn and also `software development libraries `_ like pytest, mypy, flake8, isort and black. It is optional but highly advisable that you create a virtual environment. A virtual environment -is a "separate space", where you can install Feature-engine's dependencies. To create a virtual +is a "separate space", where you can install feature-engine's dependencies. To create a virtual environment, use any virtual environment tool of your choice. Some examples include: 1. `venv `_ @@ -137,7 +137,7 @@ Install dependencies ~~~~~~~~~~~~~~~~~~~~ Now, you are ready to install all dependencies, that is, all the Python libraries used by -Feature-engine. First, navigate to your clone of Feature-engine:: +feature-engine. First, navigate to your clone of feature-engine:: $ cd feature_engine @@ -146,7 +146,7 @@ Now, install Feature_engine in developer mode:: $ pip install -e ".[docs,tests]" Don't forget the `.[docs,tests]` after the `-e`. -This will add Feature-engine to your PYTHONPATH so your code edits +This will add feature-engine to your PYTHONPATH so your code edits are automatically picked up, and there is no need to re-install the package after each code change. This will also install Feature'engine's core dependencies, as well as the dependencies needed for documentation and testing. @@ -316,7 +316,7 @@ our tests. To evaluate the code coverage, you need to run pytest with coverage:: $ coverage run -m pytest -And then, you can visualize the report like this:: +And then, you can visualise the report like this:: $ coverage report @@ -410,7 +410,7 @@ important thing is that you do not get any red warning during the build process. Using tox ~~~~~~~~~ -In Feature-engine, we use tox to run all our tests automatically. If you want to run all +In feature-engine, we use tox to run all our tests automatically. If you want to run all the tests using tox locally: 1. Install tox in your development environment:: @@ -426,7 +426,7 @@ the tests using tox locally: $ tox Just writing `tox`, will trigger automatically the functionality tests, code styling tests, -typehint tests and documentation test. These will test the entire Feature-engine ecosystem +typehint tests and documentation test. These will test the entire feature-engine ecosystem and not just your new scripts, so it will be more time consuming. If the tests pass, the code is in optimal condition :) @@ -462,13 +462,13 @@ will try to help. through the PR as well.** Once the submission is reviewed and provided the continuous integration tests have -passed and the code is up to date with Feature-engine's main branch, we will be ready -to "Squash and Merge" your contribution into the ``main`` branch of Feature-engine. +passed and the code is up to date with feature-engine's main branch, we will be ready +to "Squash and Merge" your contribution into the ``main`` branch of feature-engine. "Squash and Merge" combines all of your commits into a single commit which helps keep the history of the repository clean and tidy. Once your contribution has been merged into main, you will be listed as a -Feature-engine contributor :) +feature-engine contributor :) Merge Pull Requests @@ -486,7 +486,7 @@ Releases After a few features have been added to the main branch by yourself and other contributors, we will merge main into a release branch, e.g. 1.2.X, to release a new -version of Feature-engine to PyPI and conda-forge. +version of feature-engine to PyPI and conda-forge. Keep your Fork up to Date @@ -520,7 +520,7 @@ some conflicting files. You will need to resolve these conflicts and continue th $ git push -f origin main The previous command will update your fork (remote) so that your fork's main branch is in sync with -Feature-engine's main. Now, you need to rebase main onto your feature branch. +feature-engine's main. Now, you need to rebase main onto your feature branch. 4. Check out your feature branch:: diff --git a/docs/contribute/contribute_docs.rst b/docs/contribute/contribute_docs.rst index d020237ad..856e22d85 100644 --- a/docs/contribute/contribute_docs.rst +++ b/docs/contribute/contribute_docs.rst @@ -8,7 +8,7 @@ Contribute Docs If you contribute a new transformer, or enhance the functionality of a current transformer, most likely, you would have to add or update the documentation as well. -This is Feature-engine's documentation ecosystem: +This is feature-engine's documentation ecosystem: - Feature-engine documentation is built using `Sphinx `_ and is hosted on `Read the Docs `_. - We use the `pydata sphinx theme `_. @@ -20,7 +20,7 @@ To learn more about Sphinx check the `Sphinx Quickstart documentation `, you should have those installed already. Alternatively, first activate your environment. Then navigate to the root folder of -Feature-engine. And now install the requirements for the documentation:: +feature-engine. And now install the requirements for the documentation:: $ pip install -r docs/requirements.txt diff --git a/docs/contribute/contribute_jup.rst b/docs/contribute/contribute_jup.rst index 67b795704..9b7900c67 100644 --- a/docs/contribute/contribute_jup.rst +++ b/docs/contribute/contribute_jup.rst @@ -4,7 +4,7 @@ Contribute Jupyter notebooks ============================ We created a collection of Jupyter notebooks that showcase the main functionality of -Feature-engine's transformers. We link these notebooks throughout the main documentation +feature-engine's transformers. We link these notebooks throughout the main documentation to offer users more examples and details about transformers and how to use them. **Note** that the Jupyter notebooks are hosted in a separate diff --git a/docs/contribute/contribute_other.rst b/docs/contribute/contribute_other.rst index 5521bbcbd..f31f69bbb 100644 --- a/docs/contribute/contribute_other.rst +++ b/docs/contribute/contribute_other.rst @@ -8,25 +8,25 @@ Equally important to the code contributions are contributions to the documentati the gallery of examples on how to use the project, which we already discussed. However, a good project does no-one any good if people don't know about it. So spreading the word -about Feature-engine is extremely important. You will do the project a big favor if you help -us spread the word about Feature-engine. +about feature-engine is extremely important. You will do the project a big favor if you help +us spread the word about feature-engine. Here are some examples of how you could do that: Spread the word --------------- -1. Star `Feature-engine's Repository `_. -2. Share Feature-engine or any of our :ref:`blogs and videos ` on social media. -3. Write a blog about Feature-engine. -4. Give a talk about Feature-engine or mention it in one of your talks. -5. If you teach, use Feature-engine in your lectures. -6. Share Feature-engine with your colleagues. +1. Star `feature-engine's Repository `_. +2. Share feature-engine or any of our :ref:`blogs and videos ` on social media. +3. Write a blog about feature-engine. +4. Give a talk about feature-engine or mention it in one of your talks. +5. If you teach, use feature-engine in your lectures. +6. Share feature-engine with your colleagues. -If you write a blog or give a talk that is publicly available, and it features Feature-engine, +If you write a blog or give a talk that is publicly available, and it features feature-engine, let us know so we link it to the project or share it on social. -If you teach Feature-engine, give a shout-out on Twiter or LinkedIn and tag the maintainers. +If you teach feature-engine, give a shout-out on Twiter or LinkedIn and tag the maintainers. If you have other ideas on how to spread the word, let us know or update this page straightaway. diff --git a/docs/contribute/index.rst b/docs/contribute/index.rst index e89bbecfe..e7cef664f 100644 --- a/docs/contribute/index.rst +++ b/docs/contribute/index.rst @@ -21,7 +21,7 @@ Read more about `"Why Contribute to Open-Source" `_. - Improve our documentation, i.e., fix typos, improve grammar, or add more code examples. - Write a blog, tweet, or share our project with others. -- Use Feature-engine in your lectures if you teach. +- Use feature-engine in your lectures if you teach. With plenty of ways to get involved, we would be happy for you to support the project. diff --git a/docs/index.rst b/docs/index.rst index 87ddf7c3c..6a05f7e9e 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -107,7 +107,7 @@ commonly used to transform raw data. However, pandas is not compatible with skle to learn and store the feature engineering parameters. Feature-engine’s transformers wrap pandas functionality around an API that exposes `fit` and `transform` methods to learn -and store parameters from data and then use these parameters to transform the variables. Like this, Feature-engine makes +and store parameters from data and then use these parameters to transform the variables. Like this, feature-engine makes the awesome functionality available in pandas fully compatible with scikit-Learn. @@ -384,7 +384,7 @@ Feature-engine is supported by `Train in Data `_ a `Train in Data `_ is your go-to online school for mastering machine learning. We offer intermediate and advanced courses in Python programming, data science and machine -learning, taught by industry experts with extensive experience in developing, optimizing, +learning, taught by industry experts with extensive experience in developing, optimising, and deploying machine learning models in enterprise production environments. We focus on building a solid, intuitive grasp of machine learning concepts, backed by diff --git a/docs/resources/blogs.rst b/docs/resources/blogs.rst index 0117be861..b6e12cdeb 100644 --- a/docs/resources/blogs.rst +++ b/docs/resources/blogs.rst @@ -3,7 +3,7 @@ Blogs, Videos and More ====================== -Here you find articles and videos about Feature-engine and feature engineering and +Here you find articles and videos about feature-engine and feature engineering and selection in general. Blogs @@ -12,9 +12,9 @@ Blogs Feature engineering ~~~~~~~~~~~~~~~~~~~ -- `Feature-engine: A new open-source Python package for feature engineering `_. +- `feature-engine: A new open-source Python package for feature engineering `_. - `Practical Code Implementations of Feature Engineering for Machine Learning with Python `_. -- `Streamlining Feature Engineering Pipelines with Feature-engine `_. +- `Streamlining Feature Engineering Pipelines with feature-engine `_. - `Feature Engineering for Machine Learning: A comprehensive Overview `_. - `Variance Stabilizing Transformations `_. @@ -33,14 +33,14 @@ YouTube Playlists ~~~~~~~~~ -- `One minute to Feature-engine `_ -- `Feature-engine `_ +- `One minute to feature-engine `_ +- `feature-engine `_ Videos ~~~~~~ -- `Streamlining Feature Engineering Pipes with Feature-engine `_, Data Science Festival, 2020. -- `Optimising Feature Engineering Pipelines with Feature-engine `_, Pydata Cambridge 2020, from minute 51:43. +- `Streamlining Feature Engineering Pipes with feature-engine `_, Data Science Festival, 2020. +- `Optimising Feature Engineering Pipelines with feature-engine `_, Pydata Cambridge 2020, from minute 51:43. - `Feature selection in machine learning - DataTalks.Club `_, DataTalks.Club 2022. - `Feature engineering for time series forecasting `_ DataTalks.Club 2022. diff --git a/docs/resources/books.rst b/docs/resources/books.rst index 1f8f58152..4788ecf8b 100644 --- a/docs/resources/books.rst +++ b/docs/resources/books.rst @@ -1,7 +1,7 @@ Books ===== -You can learn more about how to use Feature-engine and feature engineering in general +You can learn more about how to use feature-engine and feature engineering in general in the following books: .. figure:: ../images/cookbook.png diff --git a/docs/resources/courses.rst b/docs/resources/courses.rst index 5aa6c554a..ec66f9618 100644 --- a/docs/resources/courses.rst +++ b/docs/resources/courses.rst @@ -1,7 +1,7 @@ Courses ======= -You can learn more about how to use Feature-engine and, feature engineering and feature +You can learn more about how to use feature-engine and, feature engineering and feature selection in general in the following online courses: .. figure:: ../images/feml.png diff --git a/docs/resources/index.rst b/docs/resources/index.rst index c9298fc01..f6bb54fc7 100644 --- a/docs/resources/index.rst +++ b/docs/resources/index.rst @@ -4,7 +4,7 @@ Resources ========= -Here you find learning resources to know more about Feature-engine and feature +Here you find learning resources to know more about feature-engine and feature engineering and selection in general. We have gathered online courses, books, blogs, videos, podcasts, jupyter notebook and diff --git a/docs/resources/tutorials.rst b/docs/resources/tutorials.rst index 19dd7b406..f09f81268 100644 --- a/docs/resources/tutorials.rst +++ b/docs/resources/tutorials.rst @@ -5,26 +5,26 @@ How To ------ Check our `jupyter notebooks `_ -showcasing the functionality of each Feature-engine transformer. +showcasing the functionality of each feature-engine transformer. Kaggle Kernels -------------- We also prepared Kaggle kernels with demos mixing data exploration, feature engineering, -feature creation, feature selection and hyperparameter optimization of entire pipelines. +feature creation, feature selection and hyperparameter optimisation of entire pipelines. - `Feature selection for bank customer satisfaction prediction `_ - `Feature engineering and selection for house price prediction `_ - `Feature creation for wine quality prediction `_ - `Feature engineering and model stacking for house price modelling `_ -- `Feature engineering with Feature-engine and Randomized search `_ -- `Feature engineering with Feature-engine and Grid search `_ +- `Feature engineering with feature-engine and Randomized search `_ +- `Feature engineering with feature-engine and Grid search `_ Video tutorials --------------- -You can find some videos on how to use Feature-engine in the -`Feature-engine playlist `_ +You can find some videos on how to use feature-engine in the +`feature-engine playlist `_ in Train in Data's YouTube channel. The list is a bit short at the moment, apologies. \ No newline at end of file diff --git a/docs/user_guide/pipeline/Pipeline.rst b/docs/user_guide/pipeline/Pipeline.rst index cc2a2aa79..f48251e73 100644 --- a/docs/user_guide/pipeline/Pipeline.rst +++ b/docs/user_guide/pipeline/Pipeline.rst @@ -5,7 +5,7 @@ Pipeline ======== :class:`Pipeline` facilitates the chaining together of multiple estimators into a unified sequence. This proves beneficial -as data processing frequently involves a predefined series of actions, such as feature selection, normalization, and +as data processing frequently involves a predefined series of actions, such as feature selection, normalisation, and training a machine learning model. Feature-engine's :class:`Pipeline` is different from scikit-learn's Pipeline in that our :class:`Pipeline` supports @@ -16,18 +16,17 @@ available in these transformers, to adjust the target variable to the remaining The Pipeline serves various functions in this context: - **Simplicity and encapsulation:** You need only call the `fit` and `predict` functions once on your data to fit an entire sequence of estimators. -**Hyperparameter Optimization:** +**Hyperparameter optimisation:** Grid search and random search can be performed over hyperparameters of all estimators in the pipeline simultaneously. -**Safety** +**Safety:** -Using a pipeline prevent the leakage of statistics from test data into the trained model during cross-validation, by +Using a pipeline prevents the leakage of statistics from test data into the trained model during cross-validation, by ensuring that the same data is used to fit the transformers and predictors. Pipeline functions @@ -37,16 +36,17 @@ Calling the `fit` function on the pipeline, is the same as calling `fit` on each transforming the input data and forwarding it to the subsequent step. The pipeline will have all the methods present in the final estimator within it. For instance, if the last estimator is -a classifier, the Pipeline can function as a classifier. Similarly, if the last estimator is a transformer, the pipeline inherits this functionality as well. +a classifier, the pipeline can function as a classifier. Similarly, if the last estimator is a transformer, the pipeline +inherits this functionality as well. Setting up a Pipeline --------------------- -The :class:`Pipeline` is constructed utilizing a list of (key, value) pairs, wherein the key represents the desired +The :class:`Pipeline` is constructed utilising a list of (key, value) pairs, wherein the key represents the desired name for the step, and the value denotes an estimator or a transformer object. In the following example, we set up a :class:`Pipeline` that drops missing data, then replaces categories with ordinal -numbers, and finally fits a Lasso regression model. +numbers, and finally fits a Lasso regression model. Let's start by importing the classes and functions we need: .. code:: python @@ -58,6 +58,10 @@ numbers, and finally fits a Lasso regression model. from sklearn.linear_model import Lasso +Next, we create a toy dataframe with missing data in a numerical and a categorical variable, plus a target variable: + +.. code:: python + X = pd.DataFrame( dict( x1=[2, 1, 1, 0, np.nan], @@ -66,6 +70,10 @@ numbers, and finally fits a Lasso regression model. ) y = pd.Series([1, 2, 3, 4, 5]) +Now, we assemble the 3 steps into a :class:`Pipeline`: + +.. code:: python + pipe = Pipeline( [ ("drop", DropMissingData()), @@ -73,7 +81,11 @@ numbers, and finally fits a Lasso regression model. ("lasso", Lasso(random_state=10)), ] ) - # predict + +Finally, we fit the pipeline and use it to make predictions: + +.. code:: python + pipe.fit(X, y) preds_pipe = pipe.predict(X) preds_pipe @@ -97,6 +109,8 @@ For example, this notation extracts the first step of the pipeline: pipe[:1] +This returns a new pipeline containing only the first step: + .. code:: python Pipeline(steps=[('drop', DropMissingData())]) @@ -108,6 +122,8 @@ This notation extracts the first **two** steps of the pipeline: pipe[:2] +Similarly, this returns a pipeline with the first two steps: + .. code:: python Pipeline(steps=[('drop', DropMissingData()), @@ -120,6 +136,8 @@ This notation extracts the last step of the pipeline: pipe[-1:] +And this returns a pipeline with only the last step: + .. code:: python Pipeline(steps=[('lasso', Lasso(random_state=10))]) @@ -158,7 +176,8 @@ Finding feature names in a Pipeline The :class:`Pipeline` includes a `get_feature_names_out()` method, similar to other transformers. By employing pipeline slicing, you can obtain the feature names entering each step. -Let's set up a Pipeline that adds new features to the dataset to make this more interesting: +Let's set up a Pipeline that adds new features to the dataset, to make this more interesting. As before, we start +with the imports: .. code:: python @@ -170,6 +189,10 @@ Let's set up a Pipeline that adds new features to the dataset to make this more from sklearn.linear_model import Lasso +We create the same toy dataframe and target as before: + +.. code:: python + X = pd.DataFrame( dict( x1=[2, 1, 1, 0, np.nan], @@ -178,6 +201,10 @@ Let's set up a Pipeline that adds new features to the dataset to make this more ) y = pd.Series([1, 2, 3, 4, 5]) +This time, we set up the pipeline with a one-hot encoder instead of an ordinal encoder, and fit it to the data: + +.. code:: python + pipe = Pipeline( [ ("drop", DropMissingData()), @@ -194,6 +221,8 @@ In the first step of the pipeline, no features are added, we just drop rows with pipe[:1].get_feature_names_out() +Indeed, we obtain the original variable names: + .. code:: python ['x1', 'x2'] @@ -205,6 +234,8 @@ should see the binary variables: pipe[:2].get_feature_names_out() +And that's exactly what we find: + .. code:: python ['x1', 'x2_a', 'x2_b'] @@ -217,7 +248,7 @@ Accessing nested parameters --------------------------- We can re-define, or re-set the parameters of the transformers and estimators within the pipeline. This is done under -the hood by the Grid search and random search. But in case you need to change a parameter in a step of the +the hood by grid search and random search. But in case you need to change a parameter in a step of the :class:`Pipeline`, this is how you do it: .. code:: python @@ -227,14 +258,14 @@ the hood by the Grid search and random search. But in case you need to change a Here, we changed the alpha of the lasso regression algorithm to 10. -Best use: Dropping rows during data preprocessing -------------------------------------------------- +Best use: dropping rows during data preprocessing +-------------------------------------------------- Feature-engine's :class:`Pipeline` was designed to support transformers that remove rows from the dataset, like `DropMissingData`, `OutlierTrimmer`, `LagFeatures` and `WindowFeatures`. We saw earlier in this page how to use :class:`Pipeline` with `DropMissingData`. Let's now take a look at how to -combine :class:`Pipeline` with `LagFeatures` and `WindowFeaures` to do multiple step forecasting. +combine :class:`Pipeline` with `LagFeatures` and `WindowFeatures` to do multiple step forecasting. We start by making imports: @@ -259,6 +290,8 @@ We'll use the Australia electricity demand dataset described here: Godahewa, Rakshitha, Bergmeir, Christoph, Webb, Geoff, Hyndman, Rob, & Montero-Manso, Pablo. (2021). Australian Electricity Demand Dataset (Version 1) [Data set]. Zenodo. https://doi.org/10.5281/zenodo.4659727 +Let's load the data and drop the `Industrial` column, which we won't use in this example: + .. code:: python url = "https://raw.githubusercontent.com/tidyverts/tsibbledata/master/data-raw/vic_elec/VIC2015/demand.csv" @@ -266,6 +299,10 @@ Electricity Demand Dataset (Version 1) [Data set]. Zenodo. https://doi.org/10.52 df.drop(columns=["Industrial"], inplace=True) +The date and time of each reading are stored as integers, so next, we convert them into an actual datetime index: + +.. code:: python + # Convert the integer Date to an actual date with datetime type df["date"] = df["Date"].apply( lambda x: pd.Timestamp("1899-12-30") + pd.Timedelta(x, unit="days") @@ -277,6 +314,10 @@ Electricity Demand Dataset (Version 1) [Data set]. Zenodo. https://doi.org/10.52 df.dropna(inplace=True) +Finally, we keep only the demand column, rename it, and resample the data to an hourly frequency: + +.. code:: python + # Rename columns df = df[["date_time", "OperationalLessIndustrial"]] @@ -348,7 +389,7 @@ Next, we split the data into a training set and a test set: X_test = df.loc[begin_test:] y_test = y.loc[begin_test:] -Next, we set up `LagFeatures` and `WindowFeatures` to create features from lags and windows: +Next, we set up `LagFeatures` to create 6 lagged versions of the demand variable: .. code:: python @@ -359,6 +400,9 @@ Next, we set up `LagFeatures` and `WindowFeatures` to create features from lags drop_na=True, ) +And we set up `WindowFeatures` to add the rolling 3-hour mean of the demand variable: + +.. code:: python winf = WindowFeatures( variables=["demand"], @@ -376,7 +420,7 @@ We wrap the lasso regression within the multioutput regressor to predict multipl lasso = MultiOutputRegressor(Lasso(random_state=0, max_iter=10)) -Now, we assemble the steps in the :class:`Pipeline` and fit it to the training data: +Now, we assemble the 3 steps into the :class:`Pipeline`: .. code:: python @@ -388,6 +432,10 @@ Now, we assemble the steps in the :class:`Pipeline` and fit it to the training d ] ).set_output(transform="pandas") +Finally, we fit the pipeline to the training data: + +.. code:: python + pipe.fit(X_train, y_train) We can obtain the datasets with the predictors and the targets like this: @@ -469,40 +517,15 @@ We see the 6 hr ahead energy demand prediction for each hour: To learn more about direct forecasting and how to create features, check out our courses: +- `Feature Engineering for Time Series Forecasting `_, online course. +- `Forecasting with Machine Learning `_, online course. -.. figure:: ../../images/fetsf.png - :width: 300 - :figclass: align-center - :align: left - :target: https://www.trainindata.com/p/feature-engineering-for-forecasting - - Feature Engineering for Time Series Forecasting - -.. figure:: ../../images/fwml.png - :width: 300 - :figclass: align-center - :align: right - :target: https://www.courses.trainindata.com/p/forecasting-with-machine-learning - - Forecasting with Machine Learning - -| -| -| -| -| -| -| -| -| -| - -Hyperparameter optimization +Hyperparameter optimisation --------------------------- -We can optimize the hyperparameters of the transformers and the estimators from a pipeline simultaneously. +We can optimise the hyperparameters of the transformers and the estimators from a pipeline simultaneously. -We'll start by loading the titanic dataset: +We'll start with the imports: .. code:: python @@ -515,12 +538,19 @@ We'll start by loading the titanic dataset: from sklearn.model_selection import train_test_split, GridSearchCV from sklearn.preprocessing import StandardScaler +Next, we load the titanic dataset: + +.. code:: python + X, y = load_titanic( return_X_y_frame=True, predictors_only=True, handle_missing=True, ) +And we split it into a training set and a test set: + +.. code:: python X_train, X_test, y_train, y_test = train_test_split( X, y, test_size=0.3, random_state=0, @@ -581,6 +611,8 @@ And we can see the best hyperparameters for each step: grid.best_params_ +This returns the following combination of hyperparameters: + .. code:: python {'enc__top_categories': None, @@ -593,6 +625,8 @@ And the best accuracy obtained with these hyperparameters: grid.best_score_ +Which is: + .. code:: python 0.7843822843822843 @@ -603,48 +637,9 @@ Additional resources To learn more about feature engineering and data preprocessing, including missing data imputation, outlier removal or capping, variable transformation and encoding, check out our online course and book: -.. figure:: ../../images/feml.png - :width: 300 - :figclass: align-center - :align: left - :target: https://www.trainindata.com/p/feature-engineering-for-machine-learning - - Feature Engineering for Machine Learning - -| -| -| -| -| -| -| -| -| -| - -Or read our book: - -.. figure:: ../../images/cookbook.png - :width: 200 - :figclass: align-center - :align: left - :target: https://www.packtpub.com/en-us/product/python-feature-engineering-cookbook-9781835883587 - - Python Feature Engineering Cookbook - -| -| -| -| -| -| -| -| -| -| -| -| -| +- `Feature Engineering for Machine Learning `_, online course. +- `Python Feature Engineering Cookbook `_, book. Both our book and course are suitable for beginners and more advanced data scientists -alike. By purchasing them you are supporting Sole, the main developer of Feature-engine. +alike. By purchasing them you are supporting `Sole `_, +the main developer of feature-engine. diff --git a/docs/user_guide/pipeline/index.rst b/docs/user_guide/pipeline/index.rst index 14d8d96d3..1e6e91628 100644 --- a/docs/user_guide/pipeline/index.rst +++ b/docs/user_guide/pipeline/index.rst @@ -3,7 +3,7 @@ Pipeline ======== -Feature-engine's Pipeline is equivalent to Scikit-learn's pipeline, and in addition, +Feature-engine's Pipeline is equivalent to scikit-learn's pipeline, and in addition, it accepts the method `transform_x_y`, to adjust both X and y, in those cases where rows are removed from X. diff --git a/docs/user_guide/pipeline/make_pipeline.rst b/docs/user_guide/pipeline/make_pipeline.rst index a46c30bbc..9a4b82a98 100644 --- a/docs/user_guide/pipeline/make_pipeline.rst +++ b/docs/user_guide/pipeline/make_pipeline.rst @@ -13,7 +13,7 @@ Setting up a Pipeline with make_pipeline ---------------------------------------- In the following example, we set up a `Pipeline` that drops missing data, then replaces categories with ordinal -numbers, and finally fits a Lasso regression model. +numbers, and finally fits a Lasso regression model. We start with the imports: .. code:: python @@ -25,6 +25,10 @@ numbers, and finally fits a Lasso regression model. from sklearn.linear_model import Lasso +Next, we create a toy dataframe with missing data in a numerical and a categorical variable, plus a target variable: + +.. code:: python + X = pd.DataFrame( dict( x1=[2, 1, 1, 0, np.nan], @@ -33,12 +37,20 @@ numbers, and finally fits a Lasso regression model. ) y = pd.Series([1, 2, 3, 4, 5]) +Now, we assemble the 3 steps with :class:`make_pipeline`, without naming them ourselves: + +.. code:: python + pipe = make_pipeline( DropMissingData(), OrdinalEncoder(encoding_method="arbitrary"), Lasso(random_state=10), ) - # predict + +Finally, we fit the pipeline and use it to make predictions: + +.. code:: python + pipe.fit(X, y) preds_pipe = pipe.predict(X) preds_pipe @@ -49,19 +61,21 @@ In the output we see the predictions made by the pipeline: array([2., 2.]) -The names of the pipeline were assigned automatically: +If we print the pipeline, we can check that the names of the steps were assigned automatically: .. code:: python print(pipe) +Each step took the lowercased class name as its step name: + .. code:: python Pipeline(steps=[('dropmissingdata', DropMissingData()), ('ordinalencoder', OrdinalEncoder(encoding_method='arbitrary')), ('lasso', Lasso(random_state=10))]) -The pipeline returned by :class:`make_pipeline` has exactly the same characteristics than +The pipeline returned by :class:`make_pipeline` has exactly the same characteristics as :class:`Pipeline`. Hence, for additional guidelines, check out the :class:`Pipeline` documentation. @@ -91,6 +105,8 @@ We'll use the Australia electricity demand dataset described here: Godahewa, Rakshitha, Bergmeir, Christoph, Webb, Geoff, Hyndman, Rob, & Montero-Manso, Pablo. (2021). Australian Electricity Demand Dataset (Version 1) [Data set]. Zenodo. https://doi.org/10.5281/zenodo.4659727 +Let's load the data and drop the `Industrial` column, which we won't use in this example: + .. code:: python url = "https://raw.githubusercontent.com/tidyverts/tsibbledata/master/data-raw/vic_elec/VIC2015/demand.csv" @@ -98,6 +114,10 @@ Electricity Demand Dataset (Version 1) [Data set]. Zenodo. https://doi.org/10.52 df.drop(columns=["Industrial"], inplace=True) +The date and time of each reading are stored as integers, so next, we convert them into an actual datetime index: + +.. code:: python + # Convert the integer Date to an actual date with datetime type df["date"] = df["Date"].apply( lambda x: pd.Timestamp("1899-12-30") + pd.Timedelta(x, unit="days") @@ -109,6 +129,10 @@ Electricity Demand Dataset (Version 1) [Data set]. Zenodo. https://doi.org/10.52 df.dropna(inplace=True) +Finally, we keep only the demand column, rename it, and resample the data to an hourly frequency: + +.. code:: python + # Rename columns df = df[["date_time", "OperationalLessIndustrial"]] @@ -172,7 +196,7 @@ Next, we split the data into a training set and a test set: X_test = df.loc[begin_test:] y_test = y.loc[begin_test:] -Next, we set up `LagFeatures` and `WindowFeatures` to create features from lags and windows: +Next, we set up `LagFeatures` to create 3 lagged versions of the demand variable: .. code:: python @@ -183,6 +207,9 @@ Next, we set up `LagFeatures` and `WindowFeatures` to create features from lags drop_na=True, ) +And we set up `WindowFeatures` to add the rolling 3-hour mean of the demand variable: + +.. code:: python winf = WindowFeatures( variables=["demand"], @@ -255,65 +282,19 @@ We see the 3 hr ahead energy demand prediction for each hour: 4 6646.981390 6970.501840 7308.359237 +Additional resources +-------------------- + To learn more about direct forecasting and how to create features, check out our courses: +- `Feature Engineering for Time Series Forecasting `_, online course. +- `Forecasting with Machine Learning `_, online course. + +To learn more about feature engineering more generally, check out our course and book: + +- `Feature Engineering for Machine Learning `_, online course. +- `Python Feature Engineering Cookbook `_, book. -.. figure:: ../../images/fetsf.png - :width: 300 - :figclass: align-center - :align: left - :target: https://www.trainindata.com/p/feature-engineering-for-forecasting - - Feature Engineering for Time Series Forecasting - -.. figure:: ../../images/fwml.png - :width: 300 - :figclass: align-center - :align: right - :target: https://www.courses.trainindata.com/p/forecasting-with-machine-learning - - Forecasting with Machine Learning - -.. figure:: ../../images/feml.png - :width: 300 - :figclass: align-center - :align: left - :target: https://www.trainindata.com/p/feature-engineering-for-machine-learning - - Feature Engineering for Machine Learning - -| -| -| -| -| -| -| -| -| -| - -.. figure:: ../../images/cookbook.png - :width: 200 - :figclass: align-center - :align: left - :target: https://www.packtpub.com/en-us/product/python-feature-engineering-cookbook-9781835883587 - - Python Feature Engineering Cookbook - -| -| -| -| -| -| -| -| -| -| -| -| -| - -Both our book and course are suitable for beginners and more advanced data scientists -alike. By purchasing them you are supporting Sole, the main developer of Feature-engine. \ No newline at end of file +Both our book and courses are suitable for beginners and more advanced data scientists +alike. By purchasing them you are supporting `Sole `_, +the main developer of feature-engine. diff --git a/docs/user_guide/preprocessing/MatchCategories.rst b/docs/user_guide/preprocessing/MatchCategories.rst index 5b480da74..f5711cef9 100644 --- a/docs/user_guide/preprocessing/MatchCategories.rst +++ b/docs/user_guide/preprocessing/MatchCategories.rst @@ -14,19 +14,22 @@ structure than, for example, 'str', and allowing faster grouping, mapping, and s operations on the resulting object. :class:`MatchCategories()` remembers the encodings or levels that represent each -category, and can thus can be used to ensure that the correct encoding gets -applied when passing categorical data to modeling packages that support this +category, and can thus be used to ensure that the correct encoding gets +applied when passing categorical data to modelling packages that support this dtype, or to prevent unseen categories from reaching a further transformer or estimator in a pipeline, for example. -Let's explore this with an example. First we load the Titanic dataset and split it into -a train and a test sets: +Let's explore this with an example. We start with the imports: .. code:: python from feature_engine.preprocessing import MatchCategories from feature_engine.datasets import load_titanic +Next, we load the Titanic dataset: + +.. code:: python + # Load dataset data = load_titanic( predictors_only=True, @@ -36,11 +39,15 @@ a train and a test sets: data['pclass'] = data['pclass'].astype('O') +And we split it into a train set and a test set: + +.. code:: python + # Split test and train train = data.iloc[0:1000, :] test = data.iloc[1000:, :] -Now, we set up :class:`MatchCategories()` and fit it to the train set. +Now, we set up :class:`MatchCategories()` and fit it to the train set: .. code:: python @@ -57,6 +64,8 @@ Now, we set up :class:`MatchCategories()` and fit it to the train set. # the transformer stores the mappings for categorical variables match_categories.category_dict_ +Here are the mappings learnt for each categorical variable: + .. code:: python {'pclass': Int64Index([1, 2, 3], dtype='int64'), @@ -64,83 +73,109 @@ Now, we set up :class:`MatchCategories()` and fit it to the train set. 'cabin': Index(['A', 'B', 'C', 'D', 'E', 'F', 'M', 'T'], dtype='object'), 'embarked': Index(['C', 'Missing', 'Q', 'S'], dtype='object')} -If we transform the test dataframe using the same `match_categories` object, -categorical variables will be converted to a 'category' dtype with the same -numeration (mapping from categories to integers) that was applied to the train -dataset: +To see why this matters, let's compare the order in which the categories of `embarked` +appear in the raw train and test sets. This is the order in the train set: .. code:: python - # encoding that would be gotten from the train set train.embarked.unique() +We obtain the following order: + .. code:: python array(['S', 'C', 'Missing', 'Q'], dtype=object) +And this is the order in the test set: + .. code:: python - - # encoding that would be gotten from the test set + test.embarked.unique() +Which is different from the train set: + .. code:: python array(['Q', 'S', 'C'], dtype=object) +The categories appear in a different order in each set. If we transform the dataframes +using the same `match_categories` object, categorical variables will be converted to a +'category' dtype with the same numeration (mapping from categories to integers) that was +applied to the train dataset. This is the order we obtain for the train set: + .. code:: python - - # with 'match_categories', the encoding remains the same + match_categories.transform(train).embarked.cat.categories +Which is: + .. code:: python Index(['C', 'Missing', 'Q', 'S'], dtype='object') +And this is the order we now obtain for the test set: + .. code:: python - # this will have the same encoding as the train set match_categories.transform(test).embarked.cat.categories +The 2 sets now show exactly the same category order: + .. code:: python Index(['C', 'Missing', 'Q', 'S'], dtype='object') -If some category was not present in the training data, it will not mapped -to any integer and will thus not get encoded. This behavior can be modified through the -parameter `errors`: +If some category was not present in the training data, it will not be mapped +to any integer and will thus not get encoded. This behaviour can be modified through the +parameter `errors`. Let's illustrate this with the `cabin` variable. These are the +categories present in the train set: .. code:: python - # categories present in the train data train.cabin.unique() +We obtain the following categories: + .. code:: python array(['B', 'C', 'E', 'D', 'A', 'M', 'T', 'F'], dtype=object) +And these are the categories present in the test set, which include a category, 'G', +that was not seen during training: + .. code:: python - - # categories present in the test data - 'G' is new + test.cabin.unique() +We obtain the following categories, including the unseen 'G': + .. code:: python array(['M', 'F', 'E', 'G'], dtype=object) +After transforming the train set, we obtain the same categories as before, now correctly +typed as 'category' dtype: + .. code:: python match_categories.transform(train).cabin.unique() +Which are: + .. code:: python ['B', 'C', 'E', 'D', 'A', 'M', 'T', 'F'] Categories (8, object): ['A', 'B', 'C', 'D', 'E', 'F', 'M', 'T'] +But when we transform the test set, the unseen category 'G' is not mapped to any integer, +and becomes a missing value instead: + .. code:: python - - # unseen category 'G' will not get mapped to any integer + match_categories.transform(test).cabin.unique() +We see that 'G' has been replaced by a missing value: + .. code:: python ['M', 'F', 'E', NaN] @@ -151,5 +186,5 @@ When to use the transformer ^^^^^^^^^^^^^^^^^^^^^^^^^^^ This transformer is useful when creating custom transformers for categorical columns, -as well as when passing categorical columns to modeling packages which support them +as well as when passing categorical columns to modelling packages which support them natively but leave the variable casting to the user, such as ``lightgbm`` or ``glum``. diff --git a/docs/user_guide/preprocessing/MatchVariables.rst b/docs/user_guide/preprocessing/MatchVariables.rst index 3c5af10ae..325aaafa0 100644 --- a/docs/user_guide/preprocessing/MatchVariables.rst +++ b/docs/user_guide/preprocessing/MatchVariables.rst @@ -13,14 +13,17 @@ test set lacks columns that were present in the train set, they will be added wi value determined by the user, for example np.nan. :class:`MatchVariables()` will also return the variables in the order seen in the train set. -Let's explore this with an example. First we load the Titanic dataset and split it into -a train and a test set: +Let's explore this with an example. We start with the imports: .. code:: python from feature_engine.preprocessing import MatchVariables from feature_engine.datasets import load_titanic +Next, we load the Titanic dataset: + +.. code:: python + # Load dataset data = load_titanic( predictors_only=True, @@ -29,11 +32,15 @@ a train and a test set: data['pclass'] = data['pclass'].astype('O') +And we split it into a train set and a test set: + +.. code:: python + # Split test and train train = data.iloc[0:1000, :] test = data.iloc[1000:, :] -Now, we set up :class:`MatchVariables()` and fit it to the train set. +Now, we set up :class:`MatchVariables()` and fit it to the train set: .. code:: python @@ -50,6 +57,8 @@ Now, we set up :class:`MatchVariables()` and fit it to the train set. # the transformer stores the input variables match_cols.feature_names_in_ +These are the variables in the train set, in the order in which they appear: + .. code:: python ['pclass', @@ -62,7 +71,8 @@ Now, we set up :class:`MatchVariables()` and fit it to the train set. 'cabin', 'embarked'] -Now, we drop some columns in the test set. +Now, we drop some columns in the test set, to simulate a test set that is missing +variables that were present in the train set: .. code:: python @@ -71,6 +81,8 @@ Now, we drop some columns in the test set. test_t.head() +We see that `sex` and `age` are no longer in the dataframe: + .. code:: python pclass survived sibsp parch fare cabin embarked @@ -82,7 +94,7 @@ Now, we drop some columns in the test set. If we transform the dataframe with the dropped columns using :class:`MatchVariables()`, we see that the new dataframe contains all the variables, and those that were missing -are now back in the data, with np.nan values as default. +are now back in the data, with np.nan values as default: .. code:: python @@ -91,6 +103,8 @@ are now back in the data, with np.nan values as default. test_tt.head() +Indeed, `sex` and `age` are back, filled with missing values: + .. code:: python The following variables are added to the DataFrame: ['age', 'sex'] @@ -114,6 +128,8 @@ test that, let's add some extra columns to the test set: test_t.head() +We now have 2 extra columns, `var_a` and `var_b`, that were not present in the train set: + .. code:: python pclass survived sibsp parch fare cabin embarked var_a var_b @@ -131,6 +147,9 @@ And now, we transform the data with :class:`MatchVariables()`: test_tt.head() +The transformer simultaneously added the missing columns with NA as values and removed +the additional columns from the resulting dataset: + .. code:: python The following variables are added to the DataFrame: ['age', 'sex'] @@ -142,30 +161,33 @@ And now, we transform the data with :class:`MatchVariables()`: 1003 3 1 NaN NaN 2 0 23.2500 n Q 1004 3 1 NaN NaN 0 0 7.7875 n Q -Now, the transformer simultaneously added the missing columns with NA as values and -removed the additional columns from the resulting dataset. - - However, if we look closely, the dtypes for the `sex` variable do not match. This could -cause issues if other transformations depend upon having the correct dtypes. +cause issues if other transformations depend upon having the correct dtypes. This is the +dtype in the train set: .. code:: python train.sex.dtype +Which is: + .. code:: python dtype('O') +And this is the dtype in the transformed test set: + .. code:: python test_tt.sex.dtype +Which does not match: + .. code:: python dtype('float64') -Set the `match_dtypes` parameter to `True` in order to align the dtypes as well. +Set the `match_dtypes` parameter to `True` in order to align the dtypes as well: .. code:: python @@ -174,18 +196,22 @@ Set the `match_dtypes` parameter to `True` in order to align the dtypes as well. test_ttt = match_cols_and_dtypes.transform(test_t) +We see in the messages that the `sex` dtype was changed to match that of the train set: + .. code:: python The following variables are added to the DataFrame: ['sex', 'age'] The following variables are dropped from the DataFrame: ['var_b', 'var_a'] The sex dtype is changing from float64 to object -Now the dtype matches. +Now the dtype matches: .. code:: python test_ttt.sex.dtype +Which is: + .. code:: python dtype('O') @@ -197,17 +223,17 @@ were added, removed and altered. We can switch off the messages through the para When to use the transformer ^^^^^^^^^^^^^^^^^^^^^^^^^^^ -These transformer is useful in "predict then optimize type of problems". In such cases, +This transformer is useful in "predict then optimise" type of problems. In such cases, a machine learning model is trained on a certain dataset, with certain input features. -Then, test sets are "post-processed" according to scenarios that want to be modelled. -For example, "what would have happened if the customer received an email campaign"? -where the variable "receive_campaign" would be turned from 0 -> 1. +Then, test sets are "post-processed" according to scenarios that need to be modelled. +For example, "what would have happened if the customer received an email campaign?" +Here, the variable "receive_campaign" would be turned from 0 to 1. -While creating these modelling datasets, a lot of meta data e.g., "scenario number", +While creating these modelling datasets, a lot of metadata, e.g., "scenario number", "time scenario was generated", etc, could be added to the data. Then we need to pass these data over to the model to obtain the modelled prediction. -:class:`MatchVariables()` provides an easy an elegant way to remove the additional metadeta, +:class:`MatchVariables()` provides an easy and elegant way to remove the additional metadata, while returning datasets with the input features in the correct order, allowing the different scenarios to be modelled directly inside a machine learning pipeline. diff --git a/docs/user_guide/selection/DropConstantFeatures.rst b/docs/user_guide/selection/DropConstantFeatures.rst index 3368758aa..e6e90c0c2 100644 --- a/docs/user_guide/selection/DropConstantFeatures.rst +++ b/docs/user_guide/selection/DropConstantFeatures.rst @@ -20,7 +20,7 @@ of the filter group of selection algorithms. In Python, we can find constant features by using pandas `std` or `unique` methods, and then remove them with `drop`. -With Scikit-learn, we can find and remove constant variables with `VarianceThreshold` to quickly +With scikit-learn, we can find and remove constant variables with `VarianceThreshold` to quickly reduce the number of features. `VarianceThreshold` is part of `sklearn.feature_selection`'s API. `VarianceThreshold`, however, would only work with numerical variables. Hence, we could only @@ -29,7 +29,7 @@ preprocessing just to remove redundant variables. Feature-engine introduces :class:`DropConstantFeatures()` to find and remove constant and quasi-constant features from a dataframe. :class:`DropConstantFeatures()` works with numerical, -categorical, or datetime variables. It is therefore more versatile than Scikit-learn’s transformer +categorical, or datetime variables. It is therefore more versatile than scikit-learn’s transformer because it allows us to drop all duplicate variables without the need for prior data transformations. By default, :class:`DropConstantFeatures()` drops constant variables. We also have the option @@ -41,14 +41,15 @@ it offers a straightforward way of reducing the feature subset. Be mindful, though, that depending on the context, quasi-constant variables could be useful. -**Example** +Python implementation +--------------------- -Let’s see how to use :class:`DropConstantFeatures()` by using the Titanic dataset. This dataset +Let's see how to use :class:`DropConstantFeatures()` by using the Titanic dataset. This dataset does not contain constant or quasi-constant variables, so for the sake of the demonstration, we will consider quasi-constant those features that show the same value in more than 70% of the rows. -We first load the data and separate it into a training set and a test set: +We start with the imports: .. code:: python @@ -56,12 +57,15 @@ We first load the data and separate it into a training set and a test set: from feature_engine.datasets import load_titanic from feature_engine.selection import DropConstantFeatures +Next, we load the Titanic dataset and separate it into a training set and a test set: + +.. code:: python + X, y = load_titanic( return_X_y_frame=True, handle_missing=True, ) - X_train, X_test, y_train, y_test = train_test_split( X, y, test_size=0.3, random_state=0, ) @@ -89,6 +93,8 @@ The variables to drop are stored in the attribute `features_to_drop_`: transformer.features_to_drop_ +These are the 4 features that show the same value in more than 70% of the rows: + .. code:: python ['parch', 'cabin', 'embarked', 'body'] @@ -101,6 +107,8 @@ the observations as follows: X_train['embarked'].value_counts(normalize = True) +We obtain the following proportions: + .. code:: python S 0.711790 @@ -118,6 +126,8 @@ Let's now evaluate `parch`: X_train['parch'].value_counts(normalize = True) +We obtain the following proportions: + .. code:: python 0 0.771834 @@ -181,7 +191,7 @@ We see the resulting dataframe below: 1193 Missing 686 Kingwilliamstown, Co Cork, Ireland Glens Falls... -Like sklearn, Feature-engine transformers have the `fit_transform` method that allows us +Like sklearn, feature-engine transformers have the `fit_transform` method that allows us to find and remove constant or quasi-constant variables in a single line of code for convenience. Like sklearn as well, `DropConstantFeatures()` has the `get_support()` method, which returns @@ -192,6 +202,9 @@ will be dropped. transformer.get_support() +We obtain a boolean vector with `True` for the features that will be retained, and +`False` for those that will be dropped: + .. code:: python [True, True, True, True, True, False, True, True, False, False, @@ -212,50 +225,9 @@ selection algorithms and then train a Logistic regression estimator: For more details about this and other feature selection methods check out these resources: - -.. figure:: ../../images/fsml.png - :width: 300 - :figclass: align-center - :align: left - :target: https://www.trainindata.com/p/feature-selection-for-machine-learning - - Feature Selection for Machine Learning - -| -| -| -| -| -| -| -| -| -| - -Or read our book: - -.. figure:: ../../images/fsmlbook.png - :width: 200 - :figclass: align-center - :align: left - :target: https://www.trainindata.com/p/feature-selection-in-machine-learning-book - - Feature Selection in Machine Learning - -| -| -| -| -| -| -| -| -| -| -| -| -| -| +- `Feature Selection for Machine Learning `_, online course. +- `Feature Selection in Machine Learning `_, book. Both our book and course are suitable for beginners and more advanced data scientists -alike. By purchasing them you are supporting Sole, the main developer of Feature-engine. \ No newline at end of file +alike. By purchasing them you are supporting `Sole `_, +the main developer of feature-engine. diff --git a/docs/user_guide/selection/DropCorrelatedFeatures.rst b/docs/user_guide/selection/DropCorrelatedFeatures.rst index ccb2aaadf..c83048f05 100644 --- a/docs/user_guide/selection/DropCorrelatedFeatures.rst +++ b/docs/user_guide/selection/DropCorrelatedFeatures.rst @@ -18,9 +18,11 @@ The transformer will examine all numerical variables automatically. Note that yo dataframe with categorical and datetime variables, and these will be ignored automatically. Alternatively, you can pass a list with the variables you wish to evaluate. -**Example** +Python implementation +--------------------- -Let's create a toy dataframe where 4 of the features are correlated: +Let's create a toy dataframe where 4 of the features are correlated. We start with the +imports: .. code:: python @@ -28,6 +30,11 @@ Let's create a toy dataframe where 4 of the features are correlated: from sklearn.datasets import make_classification from feature_engine.selection import DropCorrelatedFeatures +Next, we create a function to generate a dataframe with some correlated variables, and +use it to obtain our toy dataset: + +.. code:: python + # make dataframe with some correlated variables def make_data(): X, y = make_classification(n_samples=1000, @@ -66,6 +73,7 @@ The correlated feature groups are stored in the transformer's attributes: tr.correlated_feature_sets_ +We obtain 2 groups of correlated features: .. code:: python @@ -97,16 +105,20 @@ as well: tr.features_to_drop_ +These are the 4 correlated features that will be dropped from the dataset: + .. code:: python ['var_8', 'var_6', 'var_7', 'var_9'] If we now go ahead and print the transformed data, we see that the correlated features -have been removed. +have been removed: .. code:: python - print(print(Xt.head())) + print(Xt.head()) + +Below we see the resulting dataframe: .. code:: python @@ -137,50 +149,9 @@ All notebooks can be found in a `dedicated repository `_, online course. +- `Feature Selection in Machine Learning `_, book. Both our book and course are suitable for beginners and more advanced data scientists -alike. By purchasing them you are supporting Sole, the main developer of Feature-engine. \ No newline at end of file +alike. By purchasing them you are supporting `Sole `_, +the main developer of feature-engine. diff --git a/docs/user_guide/selection/DropDuplicateFeatures.rst b/docs/user_guide/selection/DropDuplicateFeatures.rst index 157689642..2af8b679e 100644 --- a/docs/user_guide/selection/DropDuplicateFeatures.rst +++ b/docs/user_guide/selection/DropDuplicateFeatures.rst @@ -15,7 +15,7 @@ records when we merge different data sources that show some variable overlap. Checking for and removing duplicate features is a standard procedure in any data analysis workflow that helps us reduce the dimension of the dataset quickly and ensure data quality. -In Python, we can find duplicate values in an attribute table very easily with Pandas. +In Python, we can find duplicate values in an attribute table very easily with pandas. Dropping those duplicate features, however, requires a few more lines of code. Feature-engine aims to accelerate the process of data validation by finding and removing @@ -29,10 +29,11 @@ works with numerical and categorical features alike. So let’s see how to set up :class:`DropDuplicateFeatures()`. -**Example** +Python implementation +--------------------- In this demo, we will use the Titanic dataset and introduce a few duplicated features -manually: +manually. We start with the imports: .. code:: python @@ -41,6 +42,10 @@ manually: from feature_engine.datasets import load_titanic from feature_engine.selection import DropDuplicateFeatures +Next, we load the Titanic dataset and manually duplicate 3 of its columns: + +.. code:: python + data = load_titanic( handle_missing=True, predictors_only=True, @@ -107,6 +112,8 @@ attribute: transformer.features_to_drop_ +These are the 3 duplicated features that will be removed: + .. code:: python {'age_dup', 'sex_dup', 'sibsp_dup'} @@ -125,17 +132,21 @@ the duplicated features are not there any more: train_t.columns +We confirm that the duplicated features are gone: + .. code:: python Index(['pclass', 'sex', 'age', 'sibsp', 'parch', 'fare', 'cabin', 'embarked'], dtype='object') The transformer also stores the groups of duplicated features, which is useful for data -analysis and validation. +analysis and validation: .. code:: python transformer.duplicated_feature_sets_ +We see each original feature grouped together with the duplicates that were found for it: + .. code:: python [{'sex', 'sex_dup'}, {'age', 'age_dup'}, {'sibsp', 'sibsp_dup'}] @@ -151,50 +162,9 @@ feature selection algorithms: For more details about this and other feature selection methods check out these resources: - -.. figure:: ../../images/fsml.png - :width: 300 - :figclass: align-center - :align: left - :target: https://www.trainindata.com/p/feature-selection-for-machine-learning - - Feature Selection for Machine Learning - -| -| -| -| -| -| -| -| -| -| - -Or read our book: - -.. figure:: ../../images/fsmlbook.png - :width: 200 - :figclass: align-center - :align: left - :target: https://www.trainindata.com/p/feature-selection-in-machine-learning-book - - Feature Selection in Machine Learning - -| -| -| -| -| -| -| -| -| -| -| -| -| -| +- `Feature Selection for Machine Learning `_, online course. +- `Feature Selection in Machine Learning `_, book. Both our book and course are suitable for beginners and more advanced data scientists -alike. By purchasing them you are supporting Sole, the main developer of Feature-engine. \ No newline at end of file +alike. By purchasing them you are supporting `Sole `_, +the main developer of feature-engine. diff --git a/docs/user_guide/selection/DropFeatures.rst b/docs/user_guide/selection/DropFeatures.rst index 09d3befb1..ce39e0984 100644 --- a/docs/user_guide/selection/DropFeatures.rst +++ b/docs/user_guide/selection/DropFeatures.rst @@ -10,7 +10,7 @@ dataframe. The user can pass a single variable as a string or list of variables dropped. :class:`DropFeatures()` offers similar functionality to `pandas.dataframe.drop `_, -but the difference is that :class:`DropFeatures()` can be integrated into a Scikit-learn +but the difference is that :class:`DropFeatures()` can be integrated into a scikit-learn pipeline. @@ -22,10 +22,11 @@ example, we obtain the variable `age` by subtracting `date_of_application` from variables in the dataset any more. Thus, we can add :class:`DropFeatures()` in the Pipeline to have these removed. -**Example** +Python implementation +--------------------- Let's see how to use :class:`DropFeatures()` in an example with the Titanic dataset. We -first load the data and separate it into train and test: +start with the imports: .. code:: python @@ -33,24 +34,27 @@ first load the data and separate it into train and test: from feature_engine.datasets import load_titanic from feature_engine.selection import DropFeatures +Next, we load the Titanic dataset and separate it into a train set and a test set: + +.. code:: python + X, y = load_titanic( return_X_y_frame=True, handle_missing=True, ) - X_train, X_test, y_train, y_test = train_test_split( X, y, test_size=0.3, random_state=0, ) - print(X_train.head()) - Now, we go ahead and print the dataset column names: .. code:: python X_train.columns +We see the following variables in the dataset: + .. code:: python Index(['pclass', 'name', 'sex', 'age', 'sibsp', 'parch', 'ticket', 'fare', @@ -86,6 +90,8 @@ been reduced: train_t.columns +The 6 variables that we indicated are no longer in the dataframe: + .. code:: python Index(['pclass', 'name', 'sex', 'age', 'cabin', 'embarked', 'boat'], dtype='object') @@ -103,50 +109,9 @@ All notebooks can be found in a `dedicated repository `_, online course. +- `Feature Selection in Machine Learning `_, book. Both our book and course are suitable for beginners and more advanced data scientists -alike. By purchasing them you are supporting Sole, the main developer of Feature-engine. \ No newline at end of file +alike. By purchasing them you are supporting `Sole `_, +the main developer of feature-engine. diff --git a/docs/user_guide/selection/DropHighPSIFeatures.rst b/docs/user_guide/selection/DropHighPSIFeatures.rst index 9be008e6f..b4ecf9bde 100644 --- a/docs/user_guide/selection/DropHighPSIFeatures.rst +++ b/docs/user_guide/selection/DropHighPSIFeatures.rst @@ -25,7 +25,7 @@ are between two different population samples. To determine the PSI, continuous features are sorted into discrete intervals, the fraction of observations per interval is then determined, and finally those values -are compared between the 2 groups, or as we call them in Feature-engine, between the +are compared between the 2 groups, or as we call them in feature-engine, between the basis and test sets, to obtain the PSI. In other words, the PSI is computed as follows: @@ -58,7 +58,7 @@ When working with the PSI it is worth highlighting the following: - The PSI is a suitable metric for numerical features (i.e., either continuous or with high cardinality). - For categorical or discrete features, the change in distributions is better assessed with Chi-squared. -he PSI is symmetric. That means that switching the order of the basis and test dataframes +The PSI is symmetric. That means that switching the order of the basis and test dataframes in the PSI calculation will lead to identical values. However, in this implementation, the interval limits used to calculate the PSI are inferred from the basis dataframe. Hence, switching the order of the dataframes will lead to different interval limits, which in turn may result in @@ -208,8 +208,8 @@ values of the dataframe index. This might be a good option if the index contains values or if splitting just based on `split_frac`. -Examples --------- +Python implementation +---------------------- The versatility of the class lies in the different options to split the input dataframe in a reference or basis data set with the "expected" distributions, and a test set which @@ -229,8 +229,7 @@ indicated proportion. The proportion is indicated in the `split_frac` parameter. the option to select a variable in `split_col` or leave it to None. In the latter, the dataframe index will be used to split. -Let's first create a toy dataframe containing 5 random variables and 1 variable -with a shift in its distribution (*var_3* in this case). +We start with the imports: .. code:: python @@ -240,6 +239,11 @@ with a shift in its distribution (*var_3* in this case). from sklearn.datasets import make_classification from feature_engine.selection import DropHighPSIFeatures +Next, we create a toy dataframe containing 5 random variables and 1 variable +with a shift in its distribution (*var_3* in this case): + +.. code:: python + # Create a dataframe with 500 observations and 6 random variables X, y = make_classification( n_samples=500, @@ -309,7 +313,7 @@ The cut-off value used to split the dataframe is stored in the `cut_off_` attrib transformer.cut_off_ -This yields the following answer +This yields the following answer: .. code:: python @@ -344,7 +348,7 @@ Yields the following result: ['var_3'] That the *var_3* feature is dropped during the procedure is illustrated when -looking at the columns from the `X_transformed` dataframe. +looking at the columns from the `X_transformed` dataframe: .. code:: python @@ -352,6 +356,10 @@ looking at the columns from the `X_transformed` dataframe. X_transformed.columns +We confirm that *var_3* is no longer in the dataframe: + +.. code:: python + Index(['var_0', 'var_1', 'var_2', 'var_4', 'var_5'], dtype='object') :class:`DropHighPSIFeatures()` also contains a `fit_transform` method that combines @@ -359,21 +367,22 @@ the `fit` and the `transform` methods. The difference in distribution between a non-shifted and a shifted distribution is clearly visible when plotting the cumulative density -function. - -For the shifted variable: +function. Let's plot the cumulative distribution for the shifted variable: .. code:: python X['above_cut_off'] = X.index > transformer.cut_off_ sns.ecdfplot(data=X, x='var_3', hue='above_cut_off') -and a non-shifted variable (for example *var_1*) +And now let's plot the cumulative distribution for a non-shifted variable (for +example *var_1*): .. code:: python sns.ecdfplot(data=X, x='var_1', hue='above_cut_off') +Below we can compare both plots. We see that the distribution of *var_3* clearly +shifts after the cut-off, whereas the distribution of *var_1* remains stable: .. image:: ../../images/PSI_distribution_case1.png @@ -396,7 +405,7 @@ their use to assess distribution shifts in the features. Let's create a toy dataframe representing the customers' characteristics of a company. This dataset contains six random variables (in -real life this are variables like age or postal code), the seniority of the customer +real life these are variables like age or postal code), the seniority of the customer (i.e. the number of months since the start of the relationship between the customer and the company) and the customer ID (i.e. the number (integer) used to identify the customer). Generally the customer ID grows @@ -405,7 +414,7 @@ customers. From the definition of the variables, we expect the *seniority* to increase with the customer ID and therefore to have a high PSI value when comparing early and -late customer, +late customers. We start with the imports: .. code:: python @@ -413,6 +422,10 @@ late customer, from sklearn.datasets import make_classification from feature_engine.selection import DropHighPSIFeatures +Next, we create the toy dataframe: + +.. code:: python + X, y = make_classification( n_samples=500, n_features=6, @@ -454,20 +467,20 @@ the `customer_id` feature, because this variable was used as a reference to spli 'var_5': 0.027545195437270797, 'seniority': 7.8688986006052035} +The `features_to_drop_` attribute confirms that *seniority* is the only feature to remove: .. code:: python transformer.features_to_drop_ -Gives +This gives: .. code:: python ['seniority'] - Executing the dataframe transformation leads to the exclusion of the *seniority* -feature but not to the exclusion of the *customer_id*. +feature but not to the exclusion of the *customer_id*: .. code:: python @@ -475,6 +488,10 @@ feature but not to the exclusion of the *customer_id*. X_transformed.columns +We confirm that *customer_id* is retained in the transformed dataframe: + +.. code:: python + Index(['var_0', 'var_1', 'var_2', 'var_3', 'var_4', 'var_5', 'customer_id'], dtype='object') @@ -487,8 +504,8 @@ we often want to determine if the distribution of a feature changes in time, for after a certain event like the start of the Covid-19 pandemic. This is how to do it. Let's create a toy dataframe with 6 random numerical variables -and two date variables. One will be use to specific the split of the dataframe -while the second one is expected to have a high PSI value. +and two date variables. One will be used to specify the split of the dataframe, +while the second one is expected to have a high PSI value. We start with the imports: .. code:: python @@ -497,6 +514,10 @@ while the second one is expected to have a high PSI value. from sklearn.datasets import make_classification from feature_engine.selection import DropHighPSIFeatures +Next, we create the toy dataframe: + +.. code:: python + X, y = make_classification( n_samples=1000, n_features=6, @@ -528,13 +549,17 @@ will be done comparing the periods up to the French revolution and after. **Important**: if the date variable is in pandas or NumPy datetime format, you may need to pass the cut_off value as `pd.to_datetime(1789-07-14)`. -The PSI values shows the *century* variables in unstable as its value is above -the 0.25 threshold. +The PSI values show the *century* variable as unstable, since its value is above +the 0.25 threshold: .. code:: python transformer.psi_values_ +Below we see the PSI value for every feature: + +.. code:: python + {'var_0': 0.0181623637463045, 'var_1': 0.10595496570984747, 'var_2': 0.05425659114295842, @@ -543,15 +568,19 @@ the 0.25 threshold. 'var_5': 0.10122468631060424, 'century': 8.272395772368412} -The class has correctly identified the feature to be dropped. +The class has correctly identified the feature to be dropped: .. code:: python transformer.features_to_drop_ +This confirms that *century* is the only feature to remove: + +.. code:: python + ['century'] -And the transform method correctly removes the feature. +And the transform method correctly removes the feature: .. code:: python @@ -559,6 +588,10 @@ And the transform method correctly removes the feature. X_transformed.columns +We confirm that *century* is no longer in the transformed dataframe: + +.. code:: python + Index(['var_0', 'var_1', 'var_2', 'var_3', 'var_4', 'var_5', 'time'], dtype='object') The difference in distribution between a non-shifted and @@ -572,7 +605,7 @@ We can plot the cumulative distribution of the shifted variable like this: X['above_cut_off'] = X.time > pd.to_datetime(transformer.cut_off_) sns.ecdfplot(data=X, x='century', hue='above_cut_off') -and the distribution of a non-shifted variable, for example *var_2*, like this: +And the distribution of a non-shifted variable, for example *var_2*, like this: .. code:: python @@ -610,8 +643,8 @@ Split passing a category value ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Let's show how to set up the transformer in this case. The example data set -contains 6 randoms variables, a categorical variable with the labels of the -different categories and 2 category related features. +contains 6 random variables, a categorical variable with the labels of the +different categories and 2 category related features. We start with the imports: .. code:: python @@ -621,6 +654,10 @@ different categories and 2 category related features. from sklearn.datasets import make_classification from feature_engine.selection import DropHighPSIFeatures +Next, we create the toy dataframe: + +.. code:: python + X, y = make_classification( n_samples=1000, n_features=6, @@ -645,12 +682,16 @@ with values that come before C, alphabetically, will be allocated to the referen transformer = DropHighPSIFeatures(split_col='group', cut_off='C') X_transformed = transformer.fit_transform(X) -The PSI values are provided in the `psi_values_` attribute. +The PSI values are provided in the `psi_values_` attribute: .. code:: python transformer.psi_values_ +Below we see the PSI value for every feature: + +.. code:: python + {'var_0': 0.06485778974895254, 'var_1': 0.03605540598761757, 'var_2': 0.040632784917352296, @@ -663,22 +704,28 @@ The PSI values are provided in the `psi_values_` attribute. From these values we see that the last 2 features should be removed. We can corroborate that in the `features_to_drop_` attribute: - .. code:: python transformer.features_to_drop_ +This confirms that *group_means* and *shifted_feature* will be dropped: + +.. code:: python + ['group_means', 'shifted_feature'] And these columns are removed from the original dataframe by the transform -method that, in the present case, has been applied through the fit_transform -method a couple of block cells above. - +method that, in the present case, has been applied through the `fit_transform` +method a couple of code blocks above: .. code:: python X_transformed.columns +We confirm that *group_means* and *shifted_feature* are no longer in the dataframe: + +.. code:: python + Index(['var_0', 'var_1', 'var_2', 'var_3', 'var_4', 'var_5', 'group'], dtype='object') @@ -698,53 +745,62 @@ only the values B and D. trans = DropHighPSIFeatures(split_col='group', cut_off=['A', 'C', 'E']) X_no_drift = trans.fit_transform(X) +The PSI values are provided in the `psi_values_` attribute: .. code:: python trans.psi_values_ - 'var_0': 0.04322345673014104, +Below we see the PSI value for every feature: + +.. code:: python + + {'var_0': 0.04322345673014104, 'var_1': 0.03534439253617049, 'var_2': 0.05220272785661243, 'var_3': 0.04550964862452317, 'var_4': 0.04492720670343145, 'var_5': 0.044886435640028144, 'group_means': 6.601444547497699, - 'shifted_features': 0.3683642099948127} - - -Here again, the object will remove the *group_means* and the *shifted_features* columns -from the dataframe. + 'shifted_feature': 0.3683642099948127} +Here again, the object will remove the *group_means* and the *shifted_feature* columns +from the dataframe: .. code:: python trans.features_to_drop_ - ['group_means', 'shifted_features'] +This confirms that *group_means* and *shifted_feature* will be dropped: -And these columns are removed from the original dataframe by the transform -method that has been applied through the `fit_transform` method. +.. code:: python + ['group_means', 'shifted_feature'] + +And these columns are removed from the original dataframe by the transform +method that has been applied through the `fit_transform` method: .. code:: python - X_transformed.columns + X_no_drift.columns - Index(['var_0', 'var_1', 'var_2', 'var_3', 'var_4', 'var_5', 'group'], dtype='object') +We confirm that *group_means* and *shifted_feature* are no longer in the dataframe: +.. code:: python + + Index(['var_0', 'var_1', 'var_2', 'var_3', 'var_4', 'var_5', 'group'], dtype='object') In the following plots, we can compare the distribution of a feature with high PSI and one with low PSI, in the different categories of the categorical variable. -With this code we plot the cumulative distribution of a feature which distribution is +With this code we plot the cumulative distribution of a feature whose distribution is different among the different categories of the variable: .. code:: python sns.ecdfplot(data=X, x='shifted_feature', hue='group') -With this code we plot the cumulative distribution of a feature which distribution is +And with this code we plot the cumulative distribution of a feature whose distribution is the same across the different categories of the categorical variable: .. code:: python @@ -763,14 +819,14 @@ In this case, the split is not done based on the number observations from `split_col` but from the number of distinct values in the reference variable indicated in `split_col`. -A real life example for this case is when dealing with groups of different sizes -like customers income classes ('1000', '2000', '3000', '4000', ...). -Split_distinct allows to control the numbers of classes in the basis and test -dataframes regardless of the number of observations in each class. +A real life example for this case is when dealing with groups of different sizes, +like customer income classes ('1000', '2000', '3000', '4000', ...). +The `split_distinct` parameter allows us to control the number of classes in the +basis and test dataframes, regardless of the number of observations in each class. -This case is illustrated in the toy data for this case. The data set contains -6 random variable and 1 income variable that is larger for one of the 6 group -defined (the F group). +This case is illustrated in the following toy dataset, which contains 6 random +variables and 1 income variable that is larger for one of the 6 groups defined +(the F group). We start with the imports: .. code:: python @@ -781,6 +837,10 @@ defined (the F group). from sklearn.datasets import make_classification from feature_engine.selection import DropHighPSIFeatures +Next, we create the toy dataframe: + +.. code:: python + X, y = make_classification( n_samples=1000, n_features=6, @@ -795,17 +855,19 @@ defined (the F group). # And an income variable that is category dependent. np.random.seed(0) - X['income'] = np.random.uniform(1000, 2000, 500).tolist() + - np.random.uniform(1250, 2250, 500).tolist() + X['income'] = ( + np.random.uniform(1000, 2000, 500).tolist() + + np.random.uniform(1250, 2250, 500).tolist() + ) - # Shuffle the dataframe to make the dataset more real life case. + # Shuffle the dataframe to make it more realistic. X = X.sample(frac=1).reset_index(drop=True) The `group` column contains 500 observations in the (A, B, C, D, E) group and 500 in the (F) group. -When we pass `split_distinct=True` when initializing +When we pass `split_distinct=True` when initialising the `DropHighPSIFeatures` object, the two dataframes used to compute the PSI will contain the same number of **unique** values in the `group` column (i.e., one dataframe will contain 300 rows associated to groups A, B and C @@ -830,17 +892,19 @@ This yields the following PSI values: 'var_5': 0.04119583769297253, 'income': 0.46191580731264914} -And we can find the feature that will be dropped, income, here: +And we can find the feature that will be dropped, `income`, here: .. code:: python transformer.features_to_drop_ - ['income'] +This confirms that *income* will be dropped: +.. code:: python -The former feature will be removed from the dataset when calling the `transform()` method. + ['income'] +The former feature will be removed from the dataset when calling the `transform()` method: .. code:: python @@ -848,6 +912,10 @@ The former feature will be removed from the dataset when calling the `transform( X_transformed.columns +We confirm that *income* is no longer in the dataframe: + +.. code:: python + Index(['var_0', 'var_1', 'var_2', 'var_3', 'var_4', 'var_5', 'group'], dtype='object') @@ -862,12 +930,14 @@ For the shifted variable (income): sns.ecdfplot(data=X, x='income', hue='group') -and a non-shifted variable (for example *var_4*) +And for a non-shifted variable (for example *var_4*): .. code:: python sns.ecdfplot(data=X, x="var_4", hue="group") +Below we can compare both plots. We see that the distribution of *income* shifts +noticeably between groups, whereas the distribution of *var_4* remains stable: .. image:: ../../images/PSI_distribution_case5.png @@ -885,49 +955,9 @@ All notebooks can be found in a `dedicated repository `_, online course. +- `Feature Selection in Machine Learning `_, book. Both our book and course are suitable for beginners and more advanced data scientists -alike. By purchasing them you are supporting Sole, the main developer of Feature-engine. \ No newline at end of file +alike. By purchasing them you are supporting `Sole `_, +the main developer of feature-engine. diff --git a/docs/user_guide/selection/MRMR.rst b/docs/user_guide/selection/MRMR.rst index 34902e07e..5fd9a9b18 100644 --- a/docs/user_guide/selection/MRMR.rst +++ b/docs/user_guide/selection/MRMR.rst @@ -51,7 +51,7 @@ F-statistic ~~~~~~~~~~~ The F-statistic determines the degree of linear association between the features and the target. -If the target is categorical, the F-statistic is calculated using Scikit-learn's `f_classif` +If the target is categorical, the F-statistic is calculated using scikit-learn's `f_classif` function. If the target is continuous, the F-statistic is determined using `f_regression`. Note that in both cases, these statistic is useful when the features are continuous. For discrete @@ -99,17 +99,17 @@ the dataset: Pearson's correlation coefficient or mutual information. Correlation ~~~~~~~~~~~ -To determine each features's redundancy, :class:`MRMR()` obtains Pearson's correlation +To determine each feature's redundancy, :class:`MRMR()` obtains Pearson's correlation coefficient between each feature and the features selected in previous rounds. Next, it takes the average of the absolute value of the coefficients. -Note that correlation assumes that all features are continuous, so this metric may returned biased +Note that correlation assumes that all features are continuous, so this metric may return biased results for categorical and discrete variables. Mutual information ~~~~~~~~~~~~~~~~~~ -To determine each features's redundancy, :class:`MRMR()` caclulates the mutual information +To determine each feature's redundancy, :class:`MRMR()` calculates the mutual information between each feature and the features selected in former iterations, and then takes the average. @@ -154,10 +154,10 @@ detects linear and non-linear associations, but for continuous features it takes compute, impacting the speed of selection of MRMR. -Python examples ---------------- +Python implementation +---------------------- -Let's see how to implement :class:`MRMR()`. We'll start by using Scikit-learn's breast cancer +Let's see how to implement :class:`MRMR()`. We'll start by using scikit-learn's breast cancer dataset. The target variable is binary, representing malignant or benign tumors. All predictor variables are continuous. @@ -341,7 +341,7 @@ In the following output we see the test set with a reduced number of features: In the final dataset we only have the "relevant features". And by relevant, we mean those with high association with the target, and low association with other features. -Since we left the parameter `'max_features'` as `None, :class:`MRMR()` selected 20% of the +Since we left the parameter `'max_features'` as `None`, :class:`MRMR()` selected 20% of the features in the training set. The training set contained 30 features, so 6 features remain after applying MRMR. @@ -358,9 +358,9 @@ correlation coefficient. In a similar way, the MRMR feature selection algorithm will compute the feature importance as the ratio between the random forest importance and Pearson's correlation coefficient. -Lets, set up :class:`MRMR()` to use a random forests classifier for the relevance. Note that we +Let's set up :class:`MRMR()` to use a random forests classifier for the relevance. Note that we need to specify a cross-validation scheme, a performance metric, and we have the option to pass -a grid with hyperparameters to optimize: +a grid with hyperparameters to optimise: .. code:: python @@ -422,7 +422,7 @@ Mutual information ~~~~~~~~~~~~~~~~~~ If we have non-linear associations and / or categorical or discrete variables, a better -option is to obtain the relevance and redundancy utilizing mutual information. +option is to obtain the relevance and redundancy utilising mutual information. The mutual information is calculated differently for numerical and categorical variables, so it is best to flag discrete features with a boolean array. @@ -495,7 +495,7 @@ examined by MRMR are retained in the transformed dataset: 3 5.6431 52.0 1 2 37.85 -122.25 4 3.8462 52.0 1 2 37.85 -122.25 -For compatibility with Scikit-learn, :class:`MRMR()` also supports the +For compatibility with scikit-learn, :class:`MRMR()` also supports the method `get_support()`: .. code:: python @@ -513,7 +513,7 @@ Considerations -------------- The maximum relevance minimum redundancy feature selection method is fast, and therefore allows -scrutinizing fairly big datasets. Computing the F-statistic is fast. That's one of the reasons +scrutinising fairly big datasets. Computing the F-statistic is fast. That's one of the reasons that made it gain popularity. @@ -522,50 +522,9 @@ Additional resources For more details about this and other feature selection methods check out these resources: - -.. figure:: ../../images/fsml.png - :width: 300 - :figclass: align-center - :align: left - :target: https://www.trainindata.com/p/feature-selection-for-machine-learning - - Feature Selection for Machine Learning - -| -| -| -| -| -| -| -| -| -| - -Or read our book: - -.. figure:: ../../images/fsmlbook.png - :width: 200 - :figclass: align-center - :align: left - :target: https://www.trainindata.com/p/feature-selection-in-machine-learning-book - - Feature Selection in Machine Learning - -| -| -| -| -| -| -| -| -| -| -| -| -| -| +- `Feature Selection for Machine Learning `_, online course. +- `Feature Selection in Machine Learning `_, book. Both our book and course are suitable for beginners and more advanced data scientists -alike. By purchasing them you are supporting Sole, the main developer of Feature-engine. \ No newline at end of file +alike. By purchasing them you are supporting `Sole `_, +the main developer of feature-engine. diff --git a/docs/user_guide/selection/ProbeFeatureSelection.rst b/docs/user_guide/selection/ProbeFeatureSelection.rst index 882d0ce76..557a462ba 100644 --- a/docs/user_guide/selection/ProbeFeatureSelection.rst +++ b/docs/user_guide/selection/ProbeFeatureSelection.rst @@ -100,11 +100,11 @@ importance than probes extracted from a binomial, poisson or discrete uniform di when using these models. -Python examples ---------------- +Python implementation +---------------------- Let's see how to use this transformer to select variables from UC Irvine's Breast Cancer -Wisconsin (Diagnostic) dataset, which can be found `here`_. We will use Scikit-learn to load +Wisconsin (Diagnostic) dataset, which can be found `here`_. We will use scikit-learn to load the dataset. This dataset concerns breast cancer diagnoses. The target variable is binary, i.e., malignant or benign. The data is solely comprised of numerical data. @@ -200,12 +200,14 @@ With :code:`fit()`, the transformer: Analysing the probes ~~~~~~~~~~~~~~~~~~~~ -In the attribute :code:`probe_features`, we find the pseudo-randomly generated variable(s): +In the attribute :code:`probe_features_`, we find the pseudo-randomly generated variable(s): .. code:: python sel.probe_features_.head() +These are the first 5 values of the probe feature: + .. code:: python gaussian_probe_0 @@ -405,7 +407,7 @@ transformed datasets: 'worst symmetry', 'worst fractal dimension'] -For compatibility with Scikit-learn selection transformers, :class:`ProbeFeatureSelection()` +For compatibility with scikit-learn selection transformers, :class:`ProbeFeatureSelection()` also supports the method `get_support()`: .. code:: python @@ -477,12 +479,14 @@ In the histograms we recognise the 5 well defined distributions: .. figure:: ../../images/probe_features.png :align: center -Let's display the importance of the random features +Let's display the importance of the random features: .. code:: python sel.feature_importances_.tail() +We obtain the importance of each of the 5 probe features: + .. code:: python gaussian_probe_0 0.004600 @@ -492,7 +496,6 @@ Let's display the importance of the random features poisson_probe_0 0.001759 dtype: float64 - We see that the binary feature has an extremely low importance, hence, when we take the average, the value is so small, that no feature will be dropped (remember random forests favouring highly cardinal features?): @@ -501,8 +504,7 @@ favouring highly cardinal features?): sel.features_to_drop_ - -The previous command returns and empty list: +The previous command returns an empty list: .. code:: python @@ -641,50 +643,9 @@ More info about this method can be found in these resources: For more details about this and other feature selection methods check out these resources: - -.. figure:: ../../images/fsml.png - :width: 300 - :figclass: align-center - :align: left - :target: https://www.trainindata.com/p/feature-selection-for-machine-learning - - Feature Selection for Machine Learning - -| -| -| -| -| -| -| -| -| -| - -Or read our book: - -.. figure:: ../../images/fsmlbook.png - :width: 200 - :figclass: align-center - :align: left - :target: https://www.trainindata.com/p/feature-selection-in-machine-learning-book - - Feature Selection in Machine Learning - -| -| -| -| -| -| -| -| -| -| -| -| -| -| +- `Feature Selection for Machine Learning `_, online course. +- `Feature Selection in Machine Learning `_, book. Both our book and course are suitable for beginners and more advanced data scientists -alike. By purchasing them you are supporting Sole, the main developer of Feature-engine. \ No newline at end of file +alike. By purchasing them you are supporting `Sole `_, +the main developer of feature-engine. diff --git a/docs/user_guide/selection/RecursiveFeatureAddition.rst b/docs/user_guide/selection/RecursiveFeatureAddition.rst index 6da1beece..ee1e2f4a0 100644 --- a/docs/user_guide/selection/RecursiveFeatureAddition.rst +++ b/docs/user_guide/selection/RecursiveFeatureAddition.rst @@ -48,11 +48,11 @@ Regarding the threshold, this parameter needs a bit of hand tuning. Higher thres return fewer features. -Python example --------------- +Python implementation +--------------------- -Let's see how to use this transformer with the diabetes dataset that comes in Scikit-learn. -First, we load the data: +Let's see how to use this transformer with the diabetes dataset that comes in scikit-learn. +We start with the imports: .. code:: python @@ -62,6 +62,10 @@ First, we load the data: from sklearn.linear_model import LinearRegression from feature_engine.selection import RecursiveFeatureAddition +Next, we load the diabetes dataset: + +.. code:: python + # load dataset X, y = load_diabetes(return_X_y=True, as_frame=True) @@ -92,10 +96,10 @@ we leave the parameter `threshold` to the default value which is 0.01. .. code:: python - # initialize linear regression estimator + # initialise linear regression estimator linear_model = LinearRegression() - # initialize feature selector + # initialise feature selector tr = RecursiveFeatureAddition(estimator=linear_model, scoring="r2", cv=3) With `fit()` the model finds the most useful features, that is, features that when added, @@ -304,7 +308,7 @@ These features were not deemed important by the RFA process: ['age', 'sex', 's2', 's3', 's4', 's6'] :class:`RecursiveFeatureAddition` also has the `get_support()` method that works exactly -like that of Scikit-learn's feature selection classes: +like that of scikit-learn's feature selection classes: .. code:: python @@ -317,57 +321,16 @@ be dropped: [False, False, True, True, True, False, False, False, True, False] -And that's it! You now now how to select features by recursively adding them to a dataset. +And that's it! You now know how to select features by recursively adding them to a dataset. Additional resources -------------------- For more details about this and other feature selection methods check out these resources: - -.. figure:: ../../images/fsml.png - :width: 300 - :figclass: align-center - :align: left - :target: https://www.trainindata.com/p/feature-selection-for-machine-learning - - Feature Selection for Machine Learning - -| -| -| -| -| -| -| -| -| -| - -Or read our book: - -.. figure:: ../../images/fsmlbook.png - :width: 200 - :figclass: align-center - :align: left - :target: https://www.trainindata.com/p/feature-selection-in-machine-learning-book - - Feature Selection in Machine Learning - -| -| -| -| -| -| -| -| -| -| -| -| -| -| +- `Feature Selection for Machine Learning `_, online course. +- `Feature Selection in Machine Learning `_, book. Both our book and course are suitable for beginners and more advanced data scientists -alike. By purchasing them you are supporting Sole, the main developer of Feature-engine. \ No newline at end of file +alike. By purchasing them you are supporting `Sole `_, +the main developer of feature-engine. diff --git a/docs/user_guide/selection/RecursiveFeatureElimination.rst b/docs/user_guide/selection/RecursiveFeatureElimination.rst index 581e5c8d0..709936693 100644 --- a/docs/user_guide/selection/RecursiveFeatureElimination.rst +++ b/docs/user_guide/selection/RecursiveFeatureElimination.rst @@ -8,17 +8,17 @@ RecursiveFeatureElimination :class:`RecursiveFeatureElimination` implements recursive feature elimination. Recursive feature elimination (RFE) is a backward feature selection process. -In Feature-engine's implementation of RFE, a feature will be kept or removed based on the -resulting change in model performance resulting of adding that feature to a -machine learning. This differs from Scikit-learn's implementation of +In feature-engine's implementation of RFE, a feature will be kept or removed based on the +resulting change in model performance caused by removing that feature from the +machine learning model. This differs from scikit-learn's implementation of `RFE `_ where a feature will be kept or removed based on the feature importance derived from a -machine learning model via it's coefficients parameters or 'feature_importances_` attribute. +machine learning model via its `coef_` or `feature_importances_` attribute. Feature-engine's implementation of RFE begins by training a model on the entire set of variables, and storing its performance value. From this same model, :class:`RecursiveFeatureElimination` derives the feature importance through the `coef_` or `feature_importances_` attributes, depending -if it is a linear model or a tree-based algorithm. These feature importance value is used +if it is a linear model or a tree-based algorithm. This feature importance value is used to sort the features by increasing performance, to determine the order in which the features will be recursively removed. The least important features are removed first. @@ -27,14 +27,14 @@ then the initial feature importance is determined by feature permutation. In the next step, :class:`RecursiveFeatureElimination` removes the least important feature and trains a new machine learning model using the remaining variables. If the performance of -this model is worse than the performance from the previus model, then, the feature is kept -(because eliminating the feature caused a drop in model performance) otherwise, it removed. +this model is worse than the performance from the previous model, then the feature is kept +(because eliminating the feature caused a drop in model performance); otherwise, it is removed. :class:`RecursiveFeatureElimination` removes now the second least important feature, trains a new model, compares its performance to the previous model, determines if it should remove or retain the feature, and moves on to the next variable until it evaluates all the features in the dataset. -Note that, in Feature-engine's implementation of RFE, the feature importance is used +Note that, in feature-engine's implementation of RFE, the feature importance is used just to rank features and thus determine the order in which the features will be eliminated. But whether to retain a feature is determined based on the decrease in the performance of the model after the feature elimination. @@ -56,12 +56,11 @@ recommended that you use the machine learning model that you finally intend to b Regarding the threshold, this parameter needs a bit of hand tuning. Higher thresholds will return fewer features. -Python example --------------- - -Let's see how to use this transformer with the diabetes dataset that comes in Scikit-learn. -First, we load the data: +Python implementation +--------------------- +Let's see how to use this transformer with the diabetes dataset that comes in scikit-learn. +We start with the imports: .. code:: python @@ -71,6 +70,10 @@ First, we load the data: from sklearn.linear_model import LinearRegression from feature_engine.selection import RecursiveFeatureElimination +Next, we load the diabetes dataset: + +.. code:: python + # load dataset X, y = load_diabetes(return_X_y=True, as_frame=True) @@ -100,10 +103,10 @@ we leave the parameter `threshold` to the default value which is 0.01. .. code:: python - # initialize linear regresion estimator + # initialise linear regression estimator linear_model = LinearRegression() - # initialize feature selector + # initialise feature selector tr = RecursiveFeatureElimination(estimator=linear_model, scoring="r2", cv=3) With `fit()` the model finds the most useful features, that is, features that when removed @@ -311,7 +314,7 @@ These features were not deemed important by the RFE process: ['age', 's3', 's4', 's6'] :class:`RecursiveFeatureElimination` also has the `get_support()` method that works exactly -like that of Scikit-learn's feature selection classes: +like that of scikit-learn's feature selection classes: .. code:: python @@ -324,7 +327,7 @@ be dropped: [False, True, True, True, True, True, False, False, True, False] -And that's it! You now now how to select features by recursively removing them to a dataset. +And that's it! You now know how to select features by recursively removing them from a dataset. Additional resources @@ -336,52 +339,9 @@ More details on recursive feature elimination in this article: For more details about this and other feature selection methods check out these resources: -For more details about this and other feature selection methods check out these resources: - - -.. figure:: ../../images/fsml.png - :width: 300 - :figclass: align-center - :align: left - :target: https://www.trainindata.com/p/feature-selection-for-machine-learning - - Feature Selection for Machine Learning - -| -| -| -| -| -| -| -| -| -| - -Or read our book: - -.. figure:: ../../images/fsmlbook.png - :width: 200 - :figclass: align-center - :align: left - :target: https://www.trainindata.com/p/feature-selection-in-machine-learning-book - - Feature Selection in Machine Learning - -| -| -| -| -| -| -| -| -| -| -| -| -| -| +- `Feature Selection for Machine Learning `_, online course. +- `Feature Selection in Machine Learning `_, book. Both our book and course are suitable for beginners and more advanced data scientists -alike. By purchasing them you are supporting Sole, the main developer of Feature-engine. \ No newline at end of file +alike. By purchasing them you are supporting `Sole `_, +the main developer of feature-engine. diff --git a/docs/user_guide/selection/SelectByInformationValue.rst b/docs/user_guide/selection/SelectByInformationValue.rst index 9af4084b1..49e99069e 100644 --- a/docs/user_guide/selection/SelectByInformationValue.rst +++ b/docs/user_guide/selection/SelectByInformationValue.rst @@ -60,8 +60,9 @@ The table below presents a general framework for using IV to determine a variabl Table taken from `listendata `_. -Example -------- +Python implementation +---------------------- + Let's see how to use this transformer to select variables from UC Irvine's credit approval data set which can be found `here`_. This dataset concerns credit card applications. All attribute names and values have been changed to meaningless symbols to protect confidentiality. @@ -150,11 +151,17 @@ predictive power. With :code:`fit()`, the transformer: - calculates the WoE for each variable - - calculates the the IV for each variable + - calculates the IV for each variable - identifies the variables that have an IV score below the threshold In the attribute :code:`variables_`, we find the variables that were evaluated: +.. code:: python + + sel.variables_ + +These are the 7 variables that were evaluated: + .. code:: python ['A1', 'A6', 'A7', 'A9', 'A10', 'A12', 'A13'] @@ -165,9 +172,19 @@ In the attribute :code:`features_to_drop_`, we find the variables that were not sel.features_to_drop_ +These are the 3 variables that do not meet the IV threshold: + +.. code:: python + ['A1', 'A12', 'A13'] -The attribute :code:`information_values_` shows the IV scores for each variable. +The attribute :code:`information_values_` shows the IV scores for each variable: + +.. code:: python + + sel.information_values_ + +Below we see the IV score for each variable: .. code:: python @@ -181,8 +198,15 @@ The attribute :code:`information_values_` shows the IV scores for each variable. We see that the transformer correctly selected the features that have an IV score greater than the :code:`threshold` which was set to 0.2. -The transformer also has the method `get_support` with similar functionality to Scikit-learn's -selectors method. If you execute `sel.get_support()`, you obtain: +The transformer also has the method `get_support` with similar functionality to scikit-learn's +selectors method: + +.. code:: python + + sel.get_support() + +If you execute the command above, you obtain a boolean vector with `True` for the features +that are retained, and `False` for those that are dropped: .. code:: python @@ -198,6 +222,8 @@ With :code:`transform()`, we can go ahead and drop the features that do not meet Xtr.head() +We see the resulting dataframe below: + .. code:: python A2 A3 A4 A5 A6 A7 A8 A9 A10 A11 A14 A15 @@ -217,6 +243,10 @@ And, finally, we can also obtain the names of the features in the final transfor sel.get_feature_names_out() +We obtain the names of the retained features: + +.. code:: python + ['A2', 'A3', 'A4', 'A5', 'A6', 'A7', 'A8', 'A9', 'A10', 'A11', 'A14', 'A15'] @@ -283,50 +313,9 @@ Additional resources For more details about this and other feature selection methods check out these resources: - -.. figure:: ../../images/fsml.png - :width: 300 - :figclass: align-center - :align: left - :target: https://www.trainindata.com/p/feature-selection-for-machine-learning - - Feature Selection for Machine Learning - -| -| -| -| -| -| -| -| -| -| - -Or read our book: - -.. figure:: ../../images/fsmlbook.png - :width: 200 - :figclass: align-center - :align: left - :target: https://www.trainindata.com/p/feature-selection-in-machine-learning-book - - Feature Selection in Machine Learning - -| -| -| -| -| -| -| -| -| -| -| -| -| -| +- `Feature Selection for Machine Learning `_, online course. +- `Feature Selection in Machine Learning `_, book. Both our book and course are suitable for beginners and more advanced data scientists -alike. By purchasing them you are supporting Sole, the main developer of Feature-engine. \ No newline at end of file +alike. By purchasing them you are supporting `Sole `_, +the main developer of feature-engine. diff --git a/docs/user_guide/selection/SelectByShuffling.rst b/docs/user_guide/selection/SelectByShuffling.rst index 59dcbb844..61043945d 100644 --- a/docs/user_guide/selection/SelectByShuffling.rst +++ b/docs/user_guide/selection/SelectByShuffling.rst @@ -24,11 +24,11 @@ The algorithm operates as follows: 6. If the performance drops beyond a predefined threshold, retain the feature. 7. Repeat steps 3-6 for each feature until all have been evaluated. -Python Example --------------- +Python implementation +--------------------- Let's see how to use :class:`SelectByShuffling()` with the diabetes dataset that comes -with Scikit-learn. First, we load the data: +with scikit-learn. We start with the imports: .. code:: python @@ -38,6 +38,10 @@ with Scikit-learn. First, we load the data: from sklearn.linear_model import LinearRegression from feature_engine.selection import SelectByShuffling +Next, we load the diabetes dataset: + +.. code:: python + X, y = load_diabetes(return_X_y=True, as_frame=True) print(X.head()) @@ -80,7 +84,7 @@ the performance drop is bigger than the mean drop caused by all features. random_state=0, ) -The `fit`()` method identifies important variables—those whose value permutations lead +The `fit()` method identifies important variables—those whose value permutations lead to a decline in model performance. The `transform()` method then removes these variables from the dataset. @@ -223,49 +227,9 @@ Additional resources For more details about this and other feature selection methods check out these resources: -.. figure:: ../../images/fsml.png - :width: 300 - :figclass: align-center - :align: left - :target: https://www.trainindata.com/p/feature-selection-for-machine-learning - - Feature Selection for Machine Learning - -| -| -| -| -| -| -| -| -| -| - -Or read our book: - -.. figure:: ../../images/fsmlbook.png - :width: 200 - :figclass: align-center - :align: left - :target: https://www.trainindata.com/p/feature-selection-in-machine-learning-book - - Feature Selection in Machine Learning - -| -| -| -| -| -| -| -| -| -| -| -| -| -| +- `Feature Selection for Machine Learning `_, online course. +- `Feature Selection in Machine Learning `_, book. Both our book and course are suitable for beginners and more advanced data scientists -alike. By purchasing them you are supporting Sole, the main developer of Feature-engine. \ No newline at end of file +alike. By purchasing them you are supporting `Sole `_, +the main developer of feature-engine. diff --git a/docs/user_guide/selection/SelectBySingleFeaturePerformance.rst b/docs/user_guide/selection/SelectBySingleFeaturePerformance.rst index da8555b1c..861cecf57 100644 --- a/docs/user_guide/selection/SelectBySingleFeaturePerformance.rst +++ b/docs/user_guide/selection/SelectBySingleFeaturePerformance.rst @@ -17,11 +17,11 @@ process works as follows: If the `threshold` parameter is set to `None`, the algorithm will select features with performance above the average of all individual features. -Python Example --------------- +Python implementation +--------------------- Let's see how to use :class:`SelectBySingleFeaturePerformance()` with the diabetes -dataset that comes with Scikit-learn. First, we load the data: +dataset that comes with scikit-learn. We start with the imports: .. code:: python @@ -31,6 +31,10 @@ dataset that comes with Scikit-learn. First, we load the data: from sklearn.linear_model import LinearRegression from feature_engine.selection import SelectBySingleFeaturePerformance +Next, we load the diabetes dataset: + +.. code:: python + X, y = load_diabetes(return_X_y=True, as_frame=True) print(X.head()) @@ -58,7 +62,7 @@ which r2 > 0.01. .. code:: python - # initialize feature selector + # initialise feature selector sel = SelectBySingleFeaturePerformance( estimator=LinearRegression(), scoring="r2", cv=3, threshold=0.01) @@ -77,11 +81,11 @@ The features that will be dropped are stored in the following attribute: sel.features_to_drop_ Only one feature will be dropped, because a linear model trained using this feature showed -an r2 smaller than 0. 1: +an r2 smaller than 0.01: .. code:: python - [sex] + ['sex'] Evaluating feature importance ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -203,53 +207,9 @@ All notebooks can be found in a `dedicated repository `_, online course. +- `Feature Selection in Machine Learning `_, book. Both our book and course are suitable for beginners and more advanced data scientists -alike. By purchasing them you are supporting Sole, the main developer of Feature-engine. \ No newline at end of file +alike. By purchasing them you are supporting `Sole `_, +the main developer of feature-engine. diff --git a/docs/user_guide/selection/SelectByTargetMeanPerformance.rst b/docs/user_guide/selection/SelectByTargetMeanPerformance.rst index 7c434508f..4a5a6d85a 100644 --- a/docs/user_guide/selection/SelectByTargetMeanPerformance.rst +++ b/docs/user_guide/selection/SelectByTargetMeanPerformance.rst @@ -95,14 +95,14 @@ Finally, if a numerical variable is truly continuous and not skewed, check that not accidentally cast as object. -Example -------- +Python implementation +--------------------- Let's see how to use this method to select variables in the Titanic dataset. This data -has a mix of numerical and categorical variables, then it is a good option to showcase -this selector. +has a mix of numerical and categorical variables, which makes it a good option to +showcase this selector. -Let's import the required libraries and classes, and prepare the titanic dataset: +Let's import the required libraries and classes: .. code:: python @@ -115,6 +115,10 @@ Let's import the required libraries and classes, and prepare the titanic dataset from feature_engine.encoding import RareLabelEncoder from feature_engine.selection import SelectByTargetMeanPerformance +Next, we load the Titanic dataset and prepare it for the demo: + +.. code:: python + data = load_titanic( handle_missing=True, predictors_only=True, @@ -187,8 +191,8 @@ With `fit()` the transformer: - replaces categories by the target mean - sorts numerical variables into equal-frequency bins - replaces bins by the target mean -- calculates the the roc-auc for each transformed variable -- selects features which roc-auc bigger than the average +- calculates the roc-auc for each transformed variable +- selects features whose roc-auc is bigger than the average In the attribute `variables_` we find the variables that were evaluated: @@ -196,6 +200,8 @@ In the attribute `variables_` we find the variables that were evaluated: sel.variables_ +These are the 8 variables that were evaluated: + .. code:: python ['pclass', 'sex', 'age', 'sibsp', 'parch', 'fare', 'cabin', 'embarked'] @@ -206,6 +212,8 @@ In the attribute `features_to_drop_` we find the variables that were not selecte sel.features_to_drop_ +These are the 4 variables that will be dropped: + .. code:: python ['age', 'sibsp', 'parch', 'embarked'] @@ -287,7 +295,7 @@ In the following image we see the feature importance: .. figure:: ../../images/target-mean-sel-std.png With this, we can get a better idea of the relationship between the features and the -target variable, based on a linear regression model. +target variable, based on the target mean value used as a proxy for prediction. Checking out the resulting dataframe ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -300,6 +308,8 @@ With `transform()` we can go ahead and drop the features: Xtr.head() +We see the resulting dataframe, with the 4 selected features, below: + .. code:: python pclass sex fare cabin @@ -316,6 +326,8 @@ data: sel.get_feature_names_out() +We obtain the names of the 4 features that were retained: + .. code:: python ['pclass', 'sex', 'fare', 'cabin'] @@ -331,52 +343,10 @@ Check also: All notebooks can be found in a `dedicated repository `_. For more details about this and other feature selection methods check out these resources: -For more details about this and other feature selection methods check out these resources: - -.. figure:: ../../images/fsml.png - :width: 300 - :figclass: align-center - :align: left - :target: https://www.trainindata.com/p/feature-selection-for-machine-learning - - Feature Selection for Machine Learning - -| -| -| -| -| -| -| -| -| -| - -Or read our book: - -.. figure:: ../../images/fsmlbook.png - :width: 200 - :figclass: align-center - :align: left - :target: https://www.trainindata.com/p/feature-selection-in-machine-learning-book - - Feature Selection in Machine Learning - -| -| -| -| -| -| -| -| -| -| -| -| -| -| +- `Feature Selection for Machine Learning `_, online course. +- `Feature Selection in Machine Learning `_, book. Both our book and course are suitable for beginners and more advanced data scientists -alike. By purchasing them you are supporting Sole, the main developer of Feature-engine. \ No newline at end of file +alike. By purchasing them you are supporting `Sole `_, +the main developer of feature-engine. diff --git a/docs/user_guide/selection/SmartCorrelatedSelection.rst b/docs/user_guide/selection/SmartCorrelatedSelection.rst index 682d1993b..ecf13b169 100644 --- a/docs/user_guide/selection/SmartCorrelatedSelection.rst +++ b/docs/user_guide/selection/SmartCorrelatedSelection.rst @@ -11,17 +11,17 @@ three, four, or even more features within the dataset. Consequently, determining features to retain and which ones to eliminate becomes a crucial consideration. Deciding which features to retain from a correlated group involves several strategies, -such us: +such as: -1. **Model Performance**: Some features returns model with higher performance than others. +1. **Model Performance**: Some features result in models with higher performance than others. 2. **Variability and Cardinality**: Features with higher variability or cardinality often provide more information about the target variable. 3. **Missing Data**: Features with less missing data are generally more reliable and informative. -4. **Correlation with Target**: Features that show a stronger correlation with the target variable are often more predictive and should be prioritized. +4. **Correlation with Target**: Features that show a stronger correlation with the target variable are often more predictive and should be prioritised. -We can apply this selection strategies out of the box with the :class:`SmartCorrelatedSelection`. +We can apply these selection strategies out of the box with :class:`SmartCorrelatedSelection`. From a group of correlated variables, the :class:`SmartCorrelatedSelection` will retain the variable with: @@ -72,8 +72,8 @@ the one with the highest absolute correlation. Variance ~~~~~~~~ -Let's see how to use :class:`SmartCorrelatedSelection` in a toy example. Let's create a -toy dataframe with 4 correlated features: +Let's see how to use :class:`SmartCorrelatedSelection` in a toy example. We start with +the imports: .. code:: python @@ -81,6 +81,10 @@ toy dataframe with 4 correlated features: from sklearn.datasets import make_classification from feature_engine.selection import SmartCorrelatedSelection +Next, we create a toy dataframe with 4 correlated features: + +.. code:: python + # make dataframe with some correlated variables def make_data(): X, y = make_classification(n_samples=1000, @@ -187,17 +191,21 @@ The features that will be removed from the dataset are stored in the following a tr.features_to_drop_ +These are the 4 features that will be dropped from the dataset: + .. code:: python ['var_6', 'var_4', 'var_9', 'var_0'] If we now go ahead and print the transformed data, we see that the correlated features -have been removed. +have been removed: .. code:: python print(Xt.head()) +Below we see the resulting dataframe: + .. code:: python var_1 var_2 var_3 var_5 var_7 var_8 var_10 \ @@ -221,9 +229,7 @@ Performance Let's now select the feature that returns a machine learning model with the highest performance, from each group. We'll use a decision tree. -We start by creating a toy dataframe: - - +We start with the imports: .. code:: python @@ -232,6 +238,10 @@ We start by creating a toy dataframe: from sklearn.tree import DecisionTreeClassifier from feature_engine.selection import SmartCorrelatedSelection +Next, we create a toy dataframe: + +.. code:: python + # make dataframe with some correlated variables def make_data(): X, y = make_classification(n_samples=1000, @@ -291,7 +301,7 @@ We can find the feature that will be retained as the key in the following attrib tr.correlated_feature_dict_ -The variables `var_0` and `var_7` will be retained, and the remaining ones will be dropped. +The variables `var_0` and `var_7` will be retained, and the remaining ones will be dropped: .. code:: python @@ -303,6 +313,8 @@ We find the variables that will be dropped in the following attribute: tr.features_to_drop_ +These are the 4 features that will be dropped from the dataset: + .. code:: python ['var_8', 'var_4', 'var_6', 'var_9'] @@ -313,6 +325,8 @@ And now we can print the resulting dataframe after the transformation: print(Xt.head()) +Below we see the resulting dataframe: + .. code:: python var_0 var_1 var_2 var_3 var_5 var_7 var_10 \ @@ -329,13 +343,15 @@ And now we can print the resulting dataframe after the transformation: 3 0.484649 4 -0.186530 -Let's examine other attributes that may be useful. Like with any Scikit-learn transformer +Let's examine other attributes that may be useful. Like with any scikit-learn transformer we can obtain the names of the features in the resulting dataframe as follows: .. code:: python tr.get_feature_names_out() +We obtain the names of the features that were retained: + .. code:: python ['var_0', 'var_1', 'var_2', 'var_3', 'var_5', 'var_7', 'var_10', 'var_11'] @@ -347,6 +363,9 @@ the dataframe: tr.get_support() +We obtain a boolean vector with `True` for the features that are retained, and `False` +for those that are dropped: + .. code:: python [True, True, True, True, False, True, False, True, False, False, True, True] @@ -358,7 +377,7 @@ Correlation with Target Finally, let's select the feature that shows the strongest correlation with the target variable from each group. -Let's create another toy dataframe with 4 features and a target variable: +We start with the imports: .. code:: python @@ -366,6 +385,10 @@ Let's create another toy dataframe with 4 features and a target variable: from sklearn.datasets import make_classification from feature_engine.selection import SmartCorrelatedSelection +Next, we create another toy dataframe with 4 features and a target variable: + +.. code:: python + # make dataframe with some correlated variables def make_data(): X, y = make_classification(n_samples=1000, @@ -415,10 +438,10 @@ Let's explore the correlated feature groups: tr.correlated_feature_sets_ Similarly to the previous examples, we see that the groups of correlated features are -slightly different. Here, what happened is that the features were ordered first based +slightly different. Here, what happened is that the features were ordered first based on their absolute correlation with the target, before carrying on the search of correlation -with the other features in the dataset. Like this, the first feature of the group is retained, -which is the one with highest correlation with the target. +with the other features in the dataset. Like this, the first feature of the group is retained, +which is the one with highest correlation with the target: .. code:: python @@ -430,7 +453,7 @@ We can find the features that will be retained as keys in the following dictiona tr.correlated_feature_dict_ -The variables `var_7` and `var_0` will be retained, and the remaining ones will be dropped. +The variables `var_7` and `var_0` will be retained, and the remaining ones will be dropped: .. code:: python @@ -442,6 +465,8 @@ We can check the correlation of the features with the target variable as follows print(X.corrwith(y).abs()) +We see the correlation of every feature with the target below: + .. code:: python var_0 0.270913 @@ -466,6 +491,8 @@ the `var_8`, which is why `var_0` was retained: print(X.corrwith(y).abs().loc[['var_0', 'var_8']]) +We confirm the difference below: + .. code:: python var_0 0.270913 @@ -478,6 +505,8 @@ The variables that will be dropped are available in the following attribute: tr.features_to_drop_ +These are the 4 features that will be dropped from the dataset: + .. code:: python ['var_4', 'var_6', 'var_9', 'var_8'] @@ -488,6 +517,8 @@ And now we can print the resulting dataframe after the transformation: print(Xt.head()) +Below we see the resulting dataframe: + .. code:: python var_0 var_1 var_2 var_3 var_5 var_7 var_10 \ @@ -510,6 +541,8 @@ We can also obtain the names of the features in the resulting dataframe as follo tr.get_feature_names_out() +We obtain the names of the features that were retained: + .. code:: python ['var_0', 'var_1', 'var_2', 'var_3', 'var_5', 'var_7', 'var_10', 'var_11'] @@ -521,6 +554,9 @@ the dataframe: tr.get_support() +We obtain a boolean vector with `True` for the features that are retained, and `False` +for those that are dropped: + .. code:: python [True, True, True, True, False, True, False, True, False, False, True, True] @@ -540,50 +576,9 @@ All notebooks can be found in a `dedicated repository `_, online course. +- `Feature Selection in Machine Learning `_, book. Both our book and course are suitable for beginners and more advanced data scientists -alike. By purchasing them you are supporting Sole, the main developer of Feature-engine. \ No newline at end of file +alike. By purchasing them you are supporting `Sole `_, +the main developer of feature-engine. diff --git a/docs/user_guide/selection/index.rst b/docs/user_guide/selection/index.rst index ddfa13790..d992cdb7d 100644 --- a/docs/user_guide/selection/index.rst +++ b/docs/user_guide/selection/index.rst @@ -9,20 +9,20 @@ Feature Selection Feature-engine's feature selection transformers identify features with low predictive performance and drop them from the dataset. Most of the feature selection algorithms -supported by Feature-engine are not yet available in other libraries. These algorithms +supported by feature-engine are not yet available in other libraries. These algorithms have been gathered from data science competitions or used in the industry. -Selection Mechanism Overview +Selection mechanism overview ---------------------------- -Feature-engine’s transformers select features based on different strategies. +Feature-engine's transformers select features based on different strategies. -The first strategy evaluates the features intrinsic characteristics, like their distributions. +The first strategy evaluates the features' intrinsic characteristics, like their distributions. For example, we can remove constant or quasi-constant features. Or we can remove features -whose distribution in unstable in time by using the Population Stability Index. +whose distribution is unstable in time by using the Population Stability Index. -A second strategy consists in determining the relationships between features. Among these, +A second strategy consists of determining the relationships between features. Among these, we can remove features that are duplicated or correlated. We can also select features based on their relationship with the target. To assess this, @@ -46,14 +46,14 @@ These methods are normally geared towards improving the overall performance of t machine learning model as well as reducing the feature space. -Selectors Characteristics Overview ----------------------------------- +Selectors characteristics overview +----------------------------------- -Some Feature-engine's selectors work with categorical variables off-the-shelf and/or allow -missing data in the variables. These gives you the opportunity to quickly screen features +Some feature-engine's selectors work with categorical variables off-the-shelf and/or allow +missing data in the variables. This gives you the opportunity to quickly screen features before jumping into any feature engineering. -In the following tables, we highlight the main Feature-engine selectors characteristics: +In the following tables, we highlight the main feature-engine selectors characteristics: Selection based on feature characteristics ------------------------------------------ @@ -86,7 +86,7 @@ Selection based on a machine learning model :class:`RecursiveFeatureAddition()` × × Adds features recursively by evaluating model performance ============================================ ======================= ============= ==================================================================================== -Selection procedures that require training a machine learning model from Scikit-learn +Selection procedures that require training a machine learning model from scikit-learn require numerical variables without missing data. Selection methods commonly used in finance @@ -99,11 +99,11 @@ Selection methods commonly used in finance :class:`SelectByInformationValue()` √ x Drops features with low information value ============================================ ======================= ============= ==================================================================================== -:class:`DropHighPSIFeatures()` allows to remove features with changes in their distribution. This is done by +:class:`DropHighPSIFeatures()` removes features with changes in their distribution. This is done by splitting the input dataframe in two parts and comparing the distribution of each feature in the two parts. The metric used to assess distribution shift is the Population Stability Index (PSI). Removing unstable features may lead to more robust models. In fields like Credit Risk Modelling, the Regulator -often requires the PSI of the final feature set to be below are given threshold. +often requires the PSI of the final feature set to be below a given threshold. Alternative feature selection methods ------------------------------------- @@ -122,12 +122,12 @@ Thus, it is suitable for both categorical and numerical variables. In its curren it does not support missing data. The :class:`ProbeFeatureSelection()` introduces random variables to the dataset, then creates a model and derives -the feature importance. It selects all variables whose importance is grater than the mean importance of the random +the feature importance. It selects all variables whose importance is greater than the mean importance of the random features. -Throughout the rest of user guide, you will find more details about each of the feature selection procedures. +Throughout the rest of the user guide, you will find more details about each of the feature selection procedures. -Feature Selection Algorithms +Feature selection algorithms ---------------------------- Click below to find more details on how to use each one of the transformers. @@ -151,55 +151,14 @@ Click below to find more details on how to use each one of the transformers. MRMR -Additional Resources +Additional resources -------------------- More details about feature selection can be found in the following resources: - -.. figure:: ../../images/fsml.png - :width: 300 - :figclass: align-center - :align: left - :target: https://www.trainindata.com/p/feature-selection-for-machine-learning - - Feature Selection for Machine Learning - -| -| -| -| -| -| -| -| -| -| - -Or read our book: - -.. figure:: ../../images/fsmlbook.png - :width: 200 - :figclass: align-center - :align: left - :target: https://www.trainindata.com/p/feature-selection-in-machine-learning-book - - Feature Selection in Machine Learning - -| -| -| -| -| -| -| -| -| -| -| -| -| -| +- `Feature Selection for Machine Learning `_, online course. +- `Feature Selection in Machine Learning `_, book. Both our book and course are suitable for beginners and more advanced data scientists -alike. By purchasing them you are supporting Sole, the main developer of Feature-engine. \ No newline at end of file +alike. By purchasing them you are supporting `Sole `_, +the main developer of feature-engine. diff --git a/docs/user_guide/timeseries/forecasting/ExpandingWindowFeatures.rst b/docs/user_guide/timeseries/forecasting/ExpandingWindowFeatures.rst index 6601e1d6c..d5a1c67c5 100644 --- a/docs/user_guide/timeseries/forecasting/ExpandingWindowFeatures.rst +++ b/docs/user_guide/timeseries/forecasting/ExpandingWindowFeatures.rst @@ -27,7 +27,7 @@ models, like linear regression. Expanding window features with pandas -------------------------------------- -In Python, we can create expanding window features by utilizing pandas method `expanding`. +In Python, we can create expanding window features by utilising pandas method `expanding`. For example, by executing: .. code:: python @@ -46,7 +46,7 @@ we would also shift the window forward with pandas method `shift`: X[["var_1", "var_2"].expanding(min_periods=3).agg(["max", "mean"]).shift(period=1) -Expanding window features with Feature-engine +Expanding window features with feature-engine ---------------------------------------------- :class:`ExpandingWindowFeatures` adds expanding window features to the dataframe. @@ -55,7 +55,7 @@ Window features are the result of applying an aggregation operation (e.g., mean, min, max, etc.) to a variable over a window of past data. When forecasting the future values of a variable, the past values of that variable are -likely to be predictive. To capitalize on the past values of a variable, we can simply +likely to be predictive. To capitalise on the past values of a variable, we can simply lag features with :class:`LagFeatures`. We can also create features that summarise the past values into a single quantity utilising :class:`ExpandingWindowFeatures`. @@ -77,14 +77,14 @@ from 2 weeks ago and before that, then we should lag the window feature column b :class:`ExpandingWindowFeatures` will add the new variables with a representative name to the original dataframe. It also has the methods `fit()` and `transform()` -that make it compatible with the Scikit-learn's `Pipeline` and cross-validation +that make it compatible with the scikit-learn's `Pipeline` and cross-validation functions. Note that, in the current implementation, :class:`ExpandingWindowFeatures` only works with dataframes whose index, containing the time series timestamp, contains unique values and no NaN. -Examples --------- +Python implementation +---------------------- Let's create a toy dataset to demonstrate the functionality of :class:`ExpandingWindowFeatures`. The dataframe contains 3 numerical variables, a categorical variable, and a datetime @@ -143,12 +143,17 @@ Below we see the target variable: Now we will create expanding window features from the numerical variables. In `functions`, we indicate all the operations that we want to perform over those windows. In our example below, we want to calculate the mean and the standard deviation of the -data within those windows and also find the maximum value within the windows. +data within those windows and also find the maximum value within the windows. Let's +start with the import: .. code:: python from feature_engine.timeseries.forecasting import ExpandingWindowFeatures +Now, we set up the transformer and fit it to the data: + +.. code:: python + win_f = ExpandingWindowFeatures(functions=["mean", "max", "std"]) X_tr = win_f.fit_transform(X) @@ -209,6 +214,8 @@ attribute of the :class:`ExpandingWindowFeatures`. win_f.variables_ +These are the 3 numerical variables used to create the expanding window features: + .. code:: python ['ambient_temp', 'module_temp', 'irradiation'] @@ -220,6 +227,8 @@ We can obtain the names of the variables in the returned dataframe using the win_f.get_feature_names_out() +We see the original variables, followed by the new expanding window features: + .. code:: python ['ambient_temp', @@ -261,14 +270,14 @@ We see that the resulting dataframe contains less rows than the original datafra .. code:: python - (8, 4), (8,), (6, 13), (6,)) + ((8, 4), (8,), (6, 13), (6,)) Imputing rows with nan ~~~~~~~~~~~~~~~~~~~~~~ If instead of removing the row with nan in the expanding window features, we want to impute those -values, we can do so with any of Feature-engine's imputers. Here, we will replace nan with +values, we can do so with any of feature-engine's imputers. Here, we will replace nan with the median value of the resulting window features, using the `MeanMedianImputer` within a pipeline: @@ -334,6 +343,8 @@ The following is a pandas Series: X['ambient_temp'] +This is the resulting output: + .. code:: python 2020-05-15 12:00:00 31.31 @@ -358,6 +369,8 @@ it to a pandas Dataframe using the method `to_frame()`: X_tr.head() +We obtain the following dataframe with the 2 new expanding window features: + .. code:: python ambient_temp ambient_temp_expanding_mean \ @@ -389,6 +402,8 @@ just need to remember to drop the original series after the transformation: X_tr.head() +The original variable is no longer in the output dataframe: + .. code:: python ambient_temp_expanding_mean ambient_temp_expanding_max @@ -412,6 +427,8 @@ We can easily obtain the name of the original and new variables with the method win_f.get_feature_names_out() +We obtain the names of the original variables plus the new expanding window features: + .. code:: python ['ambient_temp', @@ -435,46 +452,12 @@ Tutorials and courses For tutorials about this and other feature engineering methods for time series forecasting check out our online courses: -.. figure:: ../../../images/fetsf.png - :width: 300 - :figclass: align-center - :align: left - :target: https://www.trainindata.com/p/feature-engineering-for-forecasting - - Feature Engineering for Time Series Forecasting - -.. figure:: ../../../images/fwml.png - :width: 300 - :figclass: align-center - :align: left - :target: https://www.courses.trainindata.com/p/forecasting-with-machine-learning - - Forecasting with Machine Learning - -| -| -| -| -| -| -| -| -| -| -| -| -| -| -| -| -| -| -| -| - +- `Feature Engineering for Time Series Forecasting `_, online course. +- `Forecasting with Machine Learning `_, online course. Our courses are suitable for beginners and more advanced data scientists looking to forecast time series using traditional machine learning models, like linear regression or gradient boosting machines. -By purchasing them you are supporting Sole, the main developer of Feature-engine. \ No newline at end of file +By purchasing them you are supporting `Sole `_, +the main developer of feature-engine. \ No newline at end of file diff --git a/docs/user_guide/timeseries/forecasting/LagFeatures.rst b/docs/user_guide/timeseries/forecasting/LagFeatures.rst index 1a1433618..19a4837d0 100644 --- a/docs/user_guide/timeseries/forecasting/LagFeatures.rst +++ b/docs/user_guide/timeseries/forecasting/LagFeatures.rst @@ -6,7 +6,6 @@ LagFeatures =========== Lag features are commonly used in data science to forecast time series with traditional - machine learning models, like linear regression or random forests. A lag feature is a feature with information about a prior time step of the time series. @@ -51,13 +50,13 @@ same time. Also, unlike `pandas.shift`, we can only lag features forward. - First, it can create features with multiple values of k at the same time. - Second, it adds the features with a name to the original dataframe. -- Third, it has the methods `fit()` and `transform()` that make it compatible with the Scikit-learn's `Pipeline` and cross-validation functions. +- Third, it has the methods `fit()` and `transform()` that make it compatible with the scikit-learn's `Pipeline` and cross-validation functions. Note that, in the current implementation, :class:`LagFeatures` only works with dataframes whose index, containing the time series timestamp, contains unique values and no NaN. -Examples --------- +Python implementation +---------------------- Let's create a toy dataset to show how to add lag features with :class:`LagFeatures`. The dataframe contains 3 numerical variables, a categorical variable, and a datetime @@ -98,6 +97,8 @@ And here we print and show the target variable: y +This is our target variable: + .. code:: python 2020-05-15 12:00:00 1 @@ -152,6 +153,8 @@ The variables to lag are stored in the `variables_` attribute of the lag_f.variables_ +These are the 3 numerical variables that were lagged: + .. code:: python ['ambient_temp', 'module_temp', 'irradiation'] @@ -163,6 +166,8 @@ returned in the transformed dataframe using the `get_feature_names_out()` method lag_f.get_feature_names_out() +We see the original variables, followed by the new lag features: + .. code:: python ['ambient_temp', @@ -213,6 +218,8 @@ Alternatively, we can drop the rows with missing values in the lag features, lik print(X_tr.head()) +We see that the row with missing data was removed from the dataframe: + .. code:: python ambient_temp module_temp irradiation color \ @@ -291,6 +298,9 @@ We can get the names of features in the resulting dataframe as follows: lag_f.get_feature_names_out() +We see the original variables, followed by the 2 lag features created from each +numerical variable: + .. code:: python ['ambient_temp', @@ -363,7 +373,7 @@ We see that 2 rows were dropped from train set and target: Lag features based on datetime ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -We can also lag features utilizing information in the timestamp of the dataframe, which +We can also lag features utilising information in the timestamp of the dataframe, which is commonly cast as datetime. Let's for example create features by lagging 2 of the numerical variables 30 minutes @@ -496,6 +506,8 @@ The following is a pandas Series: X['ambient_temp'] +This is the resulting output: + .. code:: python 2020-05-15 12:00:00 31.31 @@ -519,6 +531,8 @@ pandas Series if we convert it to a pandas Dataframe using the method `to_frame( X_tr.head() +We obtain the following dataframe with the 3 lag features: + .. code:: python ambient_temp ambient_temp_lag_1 ambient_temp_lag_2 \ @@ -546,6 +560,8 @@ just need to remember to drop the original series after the transformation: X_tr.head() +The original variable is no longer in the output dataframe: + .. code:: python ambient_temp_lag_1 ambient_temp_lag_2 \ @@ -578,6 +594,8 @@ all the features in the output dataframe. lag_f.get_feature_names_out() +We obtain the names of the original variables plus the new lag features: + .. code:: python ['ambient_temp', @@ -595,7 +613,7 @@ all the features in the output dataframe. Determining the right lag ------------------------- -We can create multiple lag features by utilizing various lags. But how do we decide which +We can create multiple lag features by utilising various lags. But how do we decide which lag is a good lag? There are multiple ways to do this. @@ -613,6 +631,7 @@ to use features that capture seasonality and trend, and much more. Lags from the target vs lags from predictor variables ----------------------------------------------------- + Very often, we want to forecast the values of just one time series. For example, we want to forecast sales in the next month. The sales variable is our target variable, and we can create features by lagging past sales values. @@ -629,7 +648,7 @@ Check out the additional transformers to create window features through the use rolling windows (:class:`WindowFeatures`) or expanding windows (:class:`ExpandingWindowFeatures`). If you want to use :class:`LagFeatures` as part of a feature engineering pipeline, -check out Feature-engine's `Pipeline`. +check out feature-engine's `Pipeline`. Tutorials and courses --------------------- @@ -637,46 +656,12 @@ Tutorials and courses For tutorials about this and other feature engineering methods for time series forecasting check out our online courses: -.. figure:: ../../../images/fetsf.png - :width: 300 - :figclass: align-center - :align: left - :target: https://www.trainindata.com/p/feature-engineering-for-forecasting - - Feature Engineering for Time Series Forecasting - -.. figure:: ../../../images/fwml.png - :width: 300 - :figclass: align-center - :align: left - :target: https://www.courses.trainindata.com/p/forecasting-with-machine-learning - - Forecasting with Machine Learning - -| -| -| -| -| -| -| -| -| -| -| -| -| -| -| -| -| -| -| -| - +- `Feature Engineering for Time Series Forecasting `_, online course. +- `Forecasting with Machine Learning `_, online course. Our courses are suitable for beginners and more advanced data scientists looking to forecast time series using traditional machine learning models, like linear regression or gradient boosting machines. -By purchasing them you are supporting Sole, the main developer of Feature-engine. \ No newline at end of file +By purchasing them you are supporting `Sole `_, +the main developer of feature-engine. \ No newline at end of file diff --git a/docs/user_guide/timeseries/forecasting/WindowFeatures.rst b/docs/user_guide/timeseries/forecasting/WindowFeatures.rst index 55797a355..89b81d384 100644 --- a/docs/user_guide/timeseries/forecasting/WindowFeatures.rst +++ b/docs/user_guide/timeseries/forecasting/WindowFeatures.rst @@ -28,7 +28,7 @@ series data. A window feature is, then, a feature created after computing mathematical functions (e.g., mean, min, max, etc.) within a window over the past data. -In Python, we can create window features by utilizing pandas method `rolling`. For example, +In Python, we can create window features by utilising pandas method `rolling`. For example, by executing: .. code:: python @@ -45,10 +45,10 @@ algorithms, we also need to shift the window forward with pandas method `shift`: X[["var_1", "var_2"].rolling(window=3).agg(["max", "mean"]).shift(period=1) -Shifting is important to ensure that we are using values strictly in the past, respect -to the point that we want to forecast. +Shifting is important to ensure that we are using values strictly in the past, with +respect to the point that we want to forecast. -Sliding window features with Feature-engine +Sliding window features with feature-engine ------------------------------------------- :class:`WindowFeatures` can automatically create and add window features to the dataframe, by performing @@ -64,7 +64,7 @@ To create window features we need to determine a number of parameters. First, we identify the size of the window or windows in which we will perform the operations. For example, we can take the average of the variable over 3 months, or 6 weeks. -We also need to determine how far back is the window located respect to the data point we +We also need to determine how far back is the window located with respect to the data point we want to forecast. For example, I can take the average of the last 3 weeks of data to forecast this week of data, or I can take the average of the last 3 weeks of data to forecast next weeks data, leaving a gap of a window in between the window feature and the forecasting point. @@ -92,13 +92,13 @@ forward the value 2 weeks forward. :class:`WindowFeatures` will add the new features with a representative name to the original dataframe. It also has the methods `fit()` and `transform()` that make it -compatible with the Scikit-learn's `Pipeline` and cross-validation functions. +compatible with the scikit-learn's `Pipeline` and cross-validation functions. Note that, in the current implementation, :class:`WindowFeatures` only works with dataframes whose index, containing the time series timestamp, contains unique values and no NaN. -Examples --------- +Python implementation +---------------------- Let's create a time series dataset to see how to create window features with :class:`WindowFeatures`. The dataframe contains 3 numerical variables, a categorical @@ -168,12 +168,16 @@ data within those windows and also find the maximum value within the windows. With `freq="15min"` we indicate that we need to shift the calculations 15 minutes forward. In other words, we are using the data available in windows defined up to 15 minutes -before the forecasting point. +before the forecasting point. Let's start with the import: .. code:: python from feature_engine.timeseries.forecasting import WindowFeatures +Now, we set up the transformer and fit it to the data: + +.. code:: python + win_f = WindowFeatures( window=["30min", "60min"], functions=["mean", "max", "std"], freq="15min", ) @@ -315,6 +319,8 @@ attribute of the :class:`WindowFeatures`: win_f.variables_ +These are the 3 numerical variables used to create the window features: + .. code:: python ['ambient_temp', 'module_temp', 'irradiation'] @@ -326,6 +332,8 @@ We can obtain the names of the variables in the transformed dataframe using the win_f.get_feature_names_out() +We see the original variables, followed by the new window features: + .. code:: python ['ambient_temp', @@ -385,7 +393,7 @@ Imputing rows with nan ~~~~~~~~~~~~~~~~~~~~~~ If instead of removing the row with nan in the window features, we want to impute those -values, we can do so with any of Feature-engine's imputers. Here, we will replace nan with +values, we can do so with any of feature-engine's imputers. Here, we will replace nan with the arbitrary value -99, using the `ArbitraryNumberImputer` within a pipeline: @@ -474,6 +482,8 @@ The following is a pandas Series: X['ambient_temp'] +This is the resulting output: + .. code:: python 2020-05-15 12:00:00 31.31 @@ -502,6 +512,8 @@ to a pandas Dataframe using the method `to_frame()`: X_tr.head() +We obtain the following dataframe with the 2 new window features: + .. code:: python ambient_temp ambient_temp_window_45min_mean \ @@ -535,6 +547,8 @@ just need to remember to drop the original series after the transformation: X_tr.head() +The original variable is no longer in the output dataframe: + .. code:: python ambient_temp_window_45min_mean \ @@ -568,6 +582,8 @@ all the features in the output dataframe. win_f.get_feature_names_out() +We obtain the names of the original variables plus the new window features: + .. code:: python ['ambient_temp', @@ -623,46 +639,12 @@ Tutorials and courses For tutorials about this and other feature engineering methods for time series forecasting check out our online courses: -.. figure:: ../../../images/fetsf.png - :width: 300 - :figclass: align-center - :align: left - :target: https://www.trainindata.com/p/feature-engineering-for-forecasting - - Feature Engineering for Time Series Forecasting - -.. figure:: ../../../images/fwml.png - :width: 300 - :figclass: align-center - :align: left - :target: https://www.courses.trainindata.com/p/forecasting-with-machine-learning - - Forecasting with Machine Learning - -| -| -| -| -| -| -| -| -| -| -| -| -| -| -| -| -| -| -| -| - +- `Feature Engineering for Time Series Forecasting `_, online course. +- `Forecasting with Machine Learning `_, online course. Our courses are suitable for beginners and more advanced data scientists looking to forecast time series using traditional machine learning models, like linear regression or gradient boosting machines. -By purchasing them you are supporting Sole, the main developer of Feature-engine. \ No newline at end of file +By purchasing them you are supporting `Sole `_, +the main developer of feature-engine. \ No newline at end of file diff --git a/docs/user_guide/timeseries/forecasting/index.rst b/docs/user_guide/timeseries/forecasting/index.rst index 7ae399c4c..c05d47fee 100644 --- a/docs/user_guide/timeseries/forecasting/index.rst +++ b/docs/user_guide/timeseries/forecasting/index.rst @@ -17,7 +17,7 @@ and convert the data into a tabular format, so that it can be framed as a regres Feature-engine's time series forecasting transformers give us the ability to extract and generate useful features from time series to use for forecasting. They are built on top of Python libraries -such as Pandas and offer an interface to extract various features from temporal data simultaneously. +such as pandas and offer an interface to extract various features from temporal data simultaneously. Time series forecasting involves learning from historical data observations to predict future values. Feature-engine's offers various transformers for creating features from the past values. @@ -26,7 +26,7 @@ Feature-engine's offers various transformers for creating features from the past Lag and Window Features ----------------------- -Trend and seasonality can be captured using lag and window features. In Feature-engine, we have +Trend and seasonality can be captured using lag and window features. In feature-engine, we have three transformers to extract these features. Lag features @@ -52,7 +52,7 @@ such as rolling window or expanding window features. Window features ~~~~~~~~~~~~~~~ -:class:`WindowFeatures`, also known as rolling window features, are used to summarize past behavior over a +:class:`WindowFeatures`, also known as rolling window features, are used to summarise past behaviour over a fixed time period by computing statistics like mean, standard deviation, min, max, sum, etc. on the time series variable. For instance, in sales forecasting, calculating the "mean" sales value of the previous 4 weeks of data is a window feature. @@ -67,7 +67,7 @@ which adds more depth to the feature set. This approach helps us generate a larg and can capture both short-term and long-term patterns in historical data. To determine which window sizes or lag combinations are useful, you can either perform time series -analyses to identify relevant window sizes, or you can use Feature-engine's +analyses to identify relevant window sizes, or you can use feature-engine's :ref:`feature selection transformers `. These are used to drop subsets of variables that are uninformative and have low predictive power which in turn improves model performance. We can set aside validation data while training the @@ -89,7 +89,7 @@ This helps to capture long-term trends in the data and also model cumulative eff thus improving forecast accuracy. Expanding window features are helpful in various data science use cases like demand forecasting -for supply chain optimization, stock price prediction etc. +for supply chain optimisation, stock price prediction etc. Just like rolling window features, expanding window features can be used with various statistical methods like mean, sum, standard deviation, min, max, among others. Unlike rolling window, we @@ -99,7 +99,7 @@ don't need to specify a window size as the window is expanded automatically at e Datetime Features ----------------- -In addition to lag and window features, Feature-engine also offers transformers to extract +In addition to lag and window features, feature-engine also offers transformers to extract other attributes from the time series such as day_of_week, day_of_month, quarter, year, hour, minute etc. directly from the datetime variable using :obj:`DatetimeFeatures `. These features are important to identify seasonal patterns, daily trends, especially when certain @@ -112,7 +112,7 @@ Cyclical Features In time series data, certain time-based attributes, such as month_of_year, day_of_week, etc. are inherently cyclical. For example, after 12th month, the calendar resets to 1st month, and after the 7th weekday, the calendar resets to 1st weekday. To inform the model of this periodic structure, -Feature-engine allows us to capture this behavior through the +feature-engine allows us to capture this behaviour through the :obj:`CyclicalFeatures ` transformer. CyclicalFeatures represents datetime variables using the sine and cosine transformation, @@ -128,51 +128,15 @@ Tutorials and courses For tutorials about this and other feature engineering methods for time series forecasting check out our online courses: -.. figure:: ../../../images/fetsf.png - :width: 300 - :figclass: align-center - :align: left - :target: https://www.trainindata.com/p/feature-engineering-for-forecasting - - Feature Engineering for Time Series Forecasting - - - -.. figure:: ../../../images/fwml.png - :width: 300 - :figclass: align-center - :align: left - :target: https://www.courses.trainindata.com/p/forecasting-with-machine-learning - - Forecasting with Machine Learning - -| -| -| -| -| -| -| -| -| -| -| -| -| -| -| -| -| -| -| -| - +- `Feature Engineering for Time Series Forecasting `_, online course. +- `Forecasting with Machine Learning `_, online course. Our courses are suitable for beginners and more advanced data scientists looking to forecast time series using traditional machine learning models, like linear regression or gradient boosting machines. -By purchasing them you are supporting Sole, the main developer of Feature-engine. +By purchasing them you are supporting `Sole `_, +the main developer of feature-engine. Forecasting Features Transformers --------------------------------- diff --git a/docs/user_guide/variable_handling/check_all_variables.rst b/docs/user_guide/variable_handling/check_all_variables.rst index b53aea753..d35f7f53d 100644 --- a/docs/user_guide/variable_handling/check_all_variables.rst +++ b/docs/user_guide/variable_handling/check_all_variables.rst @@ -8,12 +8,18 @@ check_all_variables With :class:`check_all_variables()` we can check that the variables in a list are present in the dataframe. -Let's create a toy dataset with numerical, categorical and datetime variables: +Let's create a toy dataset with numerical, categorical and datetime variables. We start +with the imports: .. code:: python import pandas as pd from sklearn.datasets import make_classification + from feature_engine.variable_handling import check_all_variables + +Now, we create the dataset: + +.. code:: python X, y = make_classification( n_samples=1000, @@ -58,12 +64,10 @@ We see the resulting dataframe below: We can use :class:`check_all_variables()` with a list of variable names to verify that -the variables in the list are in the dataframe. +the variables in the list are in the dataframe: .. code:: python - from feature_engine.variable_handling import check_all_variables - checked_vars = check_all_variables(X, ["num_var_1", "cat_var1", "date1"]) checked_vars diff --git a/docs/user_guide/variable_handling/check_categorical_variables.rst b/docs/user_guide/variable_handling/check_categorical_variables.rst index 8fc6f6bce..d311decd6 100644 --- a/docs/user_guide/variable_handling/check_categorical_variables.rst +++ b/docs/user_guide/variable_handling/check_categorical_variables.rst @@ -8,12 +8,18 @@ check_categorical_variables :class:`check_categorical_variables()` checks that the variables in the list are of type object or categorical. -Let's create a toy dataset with numerical, categorical and datetime variables: +Let's create a toy dataset with numerical, categorical and datetime variables. We start +with the imports: .. code:: python import pandas as pd from sklearn.datasets import make_classification + from feature_engine.variable_handling import check_categorical_variables + +Now, we create the dataset: + +.. code:: python X, y = make_classification( n_samples=1000, @@ -57,12 +63,10 @@ We see the resulting dataframe below: 4 2020-02-24 00:04:00 2021-09-29 04:00:00 2020-02-24 -Let's now check that 3 of the variables are of type numerical: +Let's now check that 2 of the variables are of type categorical: .. code:: python - from feature_engine.variable_handling import check_categorical_variables - var_cat = check_categorical_variables(X, ["cat_var1", "date3"]) var_cat diff --git a/docs/user_guide/variable_handling/check_datetime_variables.rst b/docs/user_guide/variable_handling/check_datetime_variables.rst index 8b1f5722a..e1fb86168 100644 --- a/docs/user_guide/variable_handling/check_datetime_variables.rst +++ b/docs/user_guide/variable_handling/check_datetime_variables.rst @@ -8,12 +8,18 @@ check_datetime_variables :class:`check_datetime_variables()` checks that the variables in the list are, or can be parsed as datetime. -Let's create a toy dataset with numerical, categorical and datetime variables: +Let's create a toy dataset with numerical, categorical and datetime variables. We start +with the imports: .. code:: python import pandas as pd from sklearn.datasets import make_classification + from feature_engine.variable_handling import check_datetime_variables + +Now, we create the dataset: + +.. code:: python X, y = make_classification( n_samples=1000, @@ -63,8 +69,6 @@ Let's check that a list of variables can be parsed as datetime: .. code:: python - from feature_engine.variable_handling import check_datetime_variables - var_date = check_datetime_variables(X, ["date2", "date3"]) var_date diff --git a/docs/user_guide/variable_handling/check_numerical_variables.rst b/docs/user_guide/variable_handling/check_numerical_variables.rst index 3212be27d..795376850 100644 --- a/docs/user_guide/variable_handling/check_numerical_variables.rst +++ b/docs/user_guide/variable_handling/check_numerical_variables.rst @@ -8,11 +8,18 @@ check_numerical_variables :class:`check_numerical_variables()` checks that the variables in the list are of type numerical. -Let's create a toy dataset with numerical, categorical and datetime variables: +Let's create a toy dataset with numerical, categorical and datetime variables. We start +with the imports: .. code:: python import pandas as pd + from feature_engine.variable_handling import check_numerical_variables + +Now, we create the dataset: + +.. code:: python + df = pd.DataFrame({ "Name": ["tom", "nick", "krish", "jack"], "City": ["London", "Manchester", "Liverpool", "Bristol"], @@ -37,8 +44,6 @@ Let's now check that 2 of the variables are of type numerical: .. code:: python - from feature_engine.variable_handling import check_numerical_variables - var_num = check_numerical_variables(df, ['Age', 'Marks']) var_num diff --git a/docs/user_guide/variable_handling/find_all_variables.rst b/docs/user_guide/variable_handling/find_all_variables.rst index 08b4efb8a..055fac788 100644 --- a/docs/user_guide/variable_handling/find_all_variables.rst +++ b/docs/user_guide/variable_handling/find_all_variables.rst @@ -8,12 +8,18 @@ find_all_variables With :class:`find_all_variables()` you can automatically capture in a list the names of all the variables in the dataset. -Let's create a toy dataset with numerical, categorical and datetime variables: +Let's create a toy dataset with numerical, categorical and datetime variables. We start +with the imports: .. code:: python import pandas as pd from sklearn.datasets import make_classification + from feature_engine.variable_handling import find_all_variables + +Now, we create the dataset: + +.. code:: python X, y = make_classification( n_samples=1000, @@ -61,8 +67,6 @@ So let's do that and then display the items in the list: .. code:: python - from feature_engine.variable_handling import find_all_variables - vars_all = find_all_variables(X) vars_all diff --git a/docs/user_guide/variable_handling/find_categorical_and_numerical_variables.rst b/docs/user_guide/variable_handling/find_categorical_and_numerical_variables.rst index c7cdde83d..05d5807cc 100644 --- a/docs/user_guide/variable_handling/find_categorical_and_numerical_variables.rst +++ b/docs/user_guide/variable_handling/find_categorical_and_numerical_variables.rst @@ -9,12 +9,18 @@ With :class:`find_categorical_and_numerical_variables()` you can automatically c 2 separate lists the names of all the categorical and numerical variables in the dataset, respectively. -Let's create a toy dataset with numerical, categorical and datetime variables: +Let's create a toy dataset with numerical, categorical and datetime variables. We start +with the imports: .. code:: python import pandas as pd from sklearn.datasets import make_classification + from feature_engine.variable_handling import find_categorical_and_numerical_variables + +Now, we create the dataset: + +.. code:: python X, y = make_classification( n_samples=1000, @@ -62,8 +68,6 @@ and numerical variables in separate lists. So let's do that and then display the .. code:: python - from feature_engine.variable_handling import find_categorical_and_numerical_variables - var_cat, var_num = find_categorical_and_numerical_variables(X) var_cat, var_num @@ -91,15 +95,16 @@ We see the resulting lists below: (['cat_var1'], ['num_var_1']) -We can also select from a list of variables those that are numerical or categorical: +If the list of variables also contains variables that are neither numerical nor +categorical, like the datetime variable `date1`, these are simply ignored: .. code:: python - find_categorical_and_numerical_variables(X, ["num_var_1", "cat_var1", "date1"]) + var_cat, var_num = find_categorical_and_numerical_variables(X, ["num_var_1", "cat_var1", "date1"]) var_cat, var_num -We see the resulting lists below: +We see that `date1` is absent from the resulting lists: .. code:: python diff --git a/docs/user_guide/variable_handling/find_categorical_variables.rst b/docs/user_guide/variable_handling/find_categorical_variables.rst index 46dfd07db..a00de72af 100644 --- a/docs/user_guide/variable_handling/find_categorical_variables.rst +++ b/docs/user_guide/variable_handling/find_categorical_variables.rst @@ -8,12 +8,18 @@ find_categorical_variables With :class:`find_categorical_variables()` you can capture in a list the names of all the variables of type object, string or categorical in the dataset. -Let's create a toy dataset with numerical, categorical and datetime variables: +Let's create a toy dataset with numerical, categorical and datetime variables. We start +with the imports: .. code:: python import pandas as pd from sklearn.datasets import make_classification + from feature_engine.variable_handling import find_categorical_variables + +Now, we create the dataset: + +.. code:: python X, y = make_classification( n_samples=1000, @@ -57,14 +63,11 @@ We see the resulting dataframe below: 4 2020-02-24 00:04:00 2021-09-29 04:00:00 2020-02-24 We can use :class:`find_categorical_variables()` to capture the names of all -variables of type object, string or categorical in a list. - -So let's do that and then display the list: +variables of type object, string or categorical in a list. So let's do that and +then display the list: .. code:: python - from feature_engine.variable_handling import find_categorical_variables - var_cat = find_categorical_variables(X) var_cat @@ -86,4 +89,4 @@ a `TypeError` because the subset of the dataset contains only numerical variable To return an empty list instead of the error we need to set `return_empty` to `True` as follows: `find_categorical_variables(X[colnames], return_empty=True)`. -The previous commands returns an empty list: `[]`. +The previous command returns an empty list: `[]`. diff --git a/docs/user_guide/variable_handling/find_datetime_variables.rst b/docs/user_guide/variable_handling/find_datetime_variables.rst index 5d322c28c..443a43d4e 100644 --- a/docs/user_guide/variable_handling/find_datetime_variables.rst +++ b/docs/user_guide/variable_handling/find_datetime_variables.rst @@ -9,12 +9,18 @@ With :class:`find_datetime_variables()` you can automatically capture in a list the names of all datetime variables in a dataset, whether they are parsed as datetime or object. -Let's create a toy dataset with numerical, categorical and datetime variables: +Let's create a toy dataset with numerical, categorical and datetime variables. We start +with the imports: .. code:: python import pandas as pd from sklearn.datasets import make_classification + from feature_engine.variable_handling import find_datetime_variables + +Now, we create the dataset: + +.. code:: python X, y = make_classification( n_samples=1000, @@ -65,8 +71,6 @@ regardless of their data type. So let's do that and then display the list: .. code:: python - from feature_engine.variable_handling import find_datetime_variables - var_date = find_datetime_variables(X) var_date diff --git a/docs/user_guide/variable_handling/find_numerical_variables.rst b/docs/user_guide/variable_handling/find_numerical_variables.rst index 6a5e140d2..fcb4b40c4 100644 --- a/docs/user_guide/variable_handling/find_numerical_variables.rst +++ b/docs/user_guide/variable_handling/find_numerical_variables.rst @@ -8,11 +8,18 @@ find_numerical_variables :class:`find_numerical_variables()` returns a list with the names of the numerical variables in the dataset. -Let's create a toy dataset with numerical, categorical and datetime variables: +Let's create a toy dataset with numerical, categorical and datetime variables. We start +with the imports: .. code:: python import pandas as pd + from feature_engine.variable_handling import find_numerical_variables + +Now, we create the dataset: + +.. code:: python + df = pd.DataFrame({ "Name": ["tom", "nick", "krish", "jack"], "City": ["London", "Manchester", "Liverpool", "Bristol"], @@ -38,8 +45,6 @@ variables in a list. So let's do that and then display the list: .. code:: python - from feature_engine.variable_handling import find_numerical_variables - var_num = find_numerical_variables(df) var_num @@ -62,4 +67,4 @@ need to set `return_empty` to `True`: find_numerical_variables(df[["Name", "City", "dob"]], return_empty=True) -The previous commands returns an empty list: `[]`. +The previous command returns an empty list: `[]`. diff --git a/docs/user_guide/variable_handling/index.rst b/docs/user_guide/variable_handling/index.rst index 44e06bfd7..cf651b70b 100644 --- a/docs/user_guide/variable_handling/index.rst +++ b/docs/user_guide/variable_handling/index.rst @@ -15,7 +15,7 @@ The `retain` functions select the variables in a list if they fulfill a conditio You can use these functions to identify different sets of variables based on their data type to streamline your feature engineering pipelines or create your own -Feature-engine or Scikit-learn compatible transformers. +feature-engine or scikit-learn compatible transformers. .. toctree:: diff --git a/docs/user_guide/variable_handling/retain_variables_if_in_df.rst b/docs/user_guide/variable_handling/retain_variables_if_in_df.rst index 5552dbb2c..4f4ab71cd 100644 --- a/docs/user_guide/variable_handling/retain_variables_if_in_df.rst +++ b/docs/user_guide/variable_handling/retain_variables_if_in_df.rst @@ -8,11 +8,18 @@ retain_variables_if_in_df :class:`retain_variables_if_in_df()` returns the subset of variables in a list that is present in the dataset. -Let's create a toy dataset with numerical, categorical and datetime variables: +Let's create a toy dataset with numerical, categorical and datetime variables. We start +with the imports: .. code:: python import pandas as pd + from feature_engine.variable_handling import retain_variables_if_in_df + +Now, we create the dataset: + +.. code:: python + df = pd.DataFrame({ "Name": ["tom", "nick", "krish", "jack"], "City": ["London", "Manchester", "Liverpool", "Bristol"], @@ -39,11 +46,9 @@ resulting list: .. code:: python - from feature_engine.variable_handling import retain_variables_if_in_df - vars_in_df = retain_variables_if_in_df(df, variables = ["Name", "City", "Dogs"]) - var_in_df + vars_in_df We see the names of the subset of variables that are in the dataframe below: @@ -67,5 +72,5 @@ under the hood, to select those variables that were entered by the user and that remain in the dataset, before applying the selection algorithm. We've now decided to expose this function as part of the `variable_handling` module. It -might be useful, for example, if you are creating `Feature-engine` compatible selection +might be useful, for example, if you are creating `feature-engine` compatible selection transformers. diff --git a/docs/user_guide/wrappers/Wrapper.rst b/docs/user_guide/wrappers/Wrapper.rst index 48e9c91f6..8d92f98ae 100644 --- a/docs/user_guide/wrappers/Wrapper.rst +++ b/docs/user_guide/wrappers/Wrapper.rst @@ -5,14 +5,14 @@ SklearnTransformerWrapper ========================= -The :class:`SklearnTransformerWrapper()` applies Scikit-learn transformers to a selected +The :class:`SklearnTransformerWrapper()` applies scikit-learn transformers to a selected group of variables. It works with transformers like the SimpleImputer, OrdinalEncoder, OneHotEncoder, KBinsDiscretizer, all scalers and also transformers for feature selection. Other transformers have not been tested, but we think it should work with most of them. The :class:`SklearnTransformerWrapper()` offers similar functionality to the `ColumnTransformer `_ -class available in Scikit-learn. They differ in the implementation to select the +class available in scikit-learn. They differ in the implementation to select the variables and the output. The :class:`SklearnTransformerWrapper()` returns a pandas dataframe with the variables @@ -21,55 +21,87 @@ in the order of the original data. The returns a Numpy array, and the order of the variables may not coincide with that of the original dataset. -In the next code snippet we show how to wrap the SimpleImputer from Scikit-learn to -impute only the selected variables. +In the next code snippet we show how to wrap the SimpleImputer from scikit-learn to +impute only the selected variables. We start with the imports: .. code:: python import pandas as pd import numpy as np from sklearn.model_selection import train_test_split + from sklearn.datasets import fetch_openml from sklearn.impute import SimpleImputer from feature_engine.wrappers import SklearnTransformerWrapper - + +Next, we load the house prices dataset and split it into a train set and a test set: + +.. code:: python + # Load dataset - data = pd.read_csv('houseprice.csv') - + data = fetch_openml( + name='house_prices', + version=1, + as_frame=True, + parser='auto', + ).frame + # Separate into train and test sets X_train, X_test, y_train, y_test = train_test_split( - data.drop(['Id', 'SalePrice'], axis=1), - data['SalePrice'], test_size=0.3, random_state=0) - + data.drop(['Id', 'SalePrice'], axis=1), + data['SalePrice'], test_size=0.3, random_state=0) + +Now, we set up the wrapper with the SimpleImputer, and fit it to the train set: + +.. code:: python + # set up the wrapper with the SimpleImputer imputer = SklearnTransformerWrapper(transformer = SimpleImputer(strategy='mean'), variables = ['LotFrontage', 'MasVnrArea']) - - # fit the wrapper + SimpleImputer + + # fit the wrapper + SimpleImputer imputer.fit(X_train) - + +Finally, we impute the missing data in the train and test sets: + +.. code:: python + # transform the data X_train = imputer.transform(X_train) X_test = imputer.transform(X_test) -In the next snippet of code we show how to wrap the StandardScaler from Scikit-learn -to standardize only the selected variables. +In the next snippet of code we show how to wrap the StandardScaler from scikit-learn +to standardise only the selected variables. We start with the imports: .. code:: python import pandas as pd import numpy as np from sklearn.model_selection import train_test_split + from sklearn.datasets import fetch_openml from sklearn.preprocessing import StandardScaler from feature_engine.wrappers import SklearnTransformerWrapper +Next, we load the house prices dataset and split it into a train set and a test set: + +.. code:: python + # Load dataset - data = pd.read_csv('houseprice.csv') + data = fetch_openml( + name='house_prices', + version=1, + as_frame=True, + parser='auto', + ).frame # Separate into train and test sets X_train, X_test, y_train, y_test = train_test_split( - data.drop(['Id', 'SalePrice'], axis=1), - data['SalePrice'], test_size=0.3, random_state=0) + data.drop(['Id', 'SalePrice'], axis=1), + data['SalePrice'], test_size=0.3, random_state=0) + +Now, we set up the wrapper with the StandardScaler, and fit it to the train set: + +.. code:: python # set up the wrapper with the StandardScaler scaler = SklearnTransformerWrapper(transformer = StandardScaler(), @@ -78,33 +110,54 @@ to standardize only the selected variables. # fit the wrapper + StandardScaler scaler.fit(X_train) +Finally, we standardise the selected variables in the train and test sets: + +.. code:: python + # transform the data X_train = scaler.transform(X_train) X_test = scaler.transform(X_test) -In the next snippet of code we show how to wrap the SelectKBest from Scikit-learn -to select only a subset of the variables. +In the next snippet of code we show how to wrap the SelectKBest from scikit-learn +to select only a subset of the variables. We start with the imports: .. code:: python import pandas as pd import numpy as np from sklearn.model_selection import train_test_split + from sklearn.datasets import fetch_openml from sklearn.feature_selection import f_regression, SelectKBest from feature_engine.wrappers import SklearnTransformerWrapper +Next, we load the house prices dataset and split it into a train set and a test set: + +.. code:: python + # Load dataset - data = pd.read_csv('houseprice.csv') + data = fetch_openml( + name='house_prices', + version=1, + as_frame=True, + parser='auto', + ).frame # Separate into train and test sets X_train, X_test, y_train, y_test = train_test_split( - data.drop(['Id', 'SalePrice'], axis=1), - data['SalePrice'], test_size=0.3, random_state=0) + data.drop(['Id', 'SalePrice'], axis=1), + data['SalePrice'], test_size=0.3, random_state=0) + +We'll apply the selector to the numerical variables, so let's capture their names in a +list: + +.. code:: python cols = [var for var in X_train.columns if X_train[var].dtypes !='O'] - # let's apply the standard scaler on the above variables +Now, we set up the wrapper with the SelectKBest, and fit it to the train set: + +.. code:: python selector = SklearnTransformerWrapper( transformer = SelectKBest(f_regression, k=5), @@ -112,15 +165,20 @@ to select only a subset of the variables. selector.fit(X_train.fillna(0), y_train) +Finally, we select the 5 best features in the train and test sets: + +.. code:: python + # transform the data X_train_t = selector.transform(X_train.fillna(0)) X_test_t = selector.transform(X_test.fillna(0)) -Even though Feature-engine has its own implementation of OneHotEncoder, you may want -to use Scikit-Learn's transformer in order to access different options, -such as drop first Category. -In the following example, we show you how to apply Scikit-learn's OneHotEncoder to a -subset of categories using the :class:SklearnTransformerWrapper(). +Even though feature-engine has its own implementation of OneHotEncoder, you may want +to use scikit-learn's transformer in order to access different options, +such as drop first category. +In the following example, we show you how to apply scikit-learn's OneHotEncoder to a +subset of categories using the :class:`SklearnTransformerWrapper()`. We start with the +imports and a function to load and clean the Titanic dataset: .. code:: python @@ -140,6 +198,10 @@ subset of categories using the :class:SklearnTransformerWrapper(). data.drop(["name", "home.dest", "ticket", "boat", "body"], axis=1, inplace=True) return data +Next, we load the data and split it into a train set and a test set: + +.. code:: python + df = load_titanic() X_train, X_test, y_train, y_test= train_test_split( @@ -149,6 +211,11 @@ subset of categories using the :class:SklearnTransformerWrapper(). random_state=42, ) +Now, we set up the wrapper with the OneHotEncoder, fit it to the train set, and use it +to transform the train and test sets: + +.. code:: python + ohe = SklearnTransformerWrapper( OneHotEncoder(sparse=False, drop='first'), variables = ['pclass','sex']) @@ -176,14 +243,14 @@ The resulting dataframe is: 147 NaN 0 0 42.4 n S 0.0 0.0 1.0 -Let's say you want to use :class:`SklearnTransformerWrapper()` in a more complex -context. As you may note there are `?` signs to denote unknown values. Due to the -complexity of the transformations needed we'll use a Pipeline to impute missing values, -encode categorical features and create interactions for specific variables using -Scikit-Learn's PolynomialFeatures. +Let's say you want to use :class:`SklearnTransformerWrapper()` in a more complex +context. As you may note there are `?` signs to denote unknown values. Due to the +complexity of the transformations needed we'll use a Pipeline to impute missing values, +encode categorical features and create interactions for specific variables using +scikit-learn's PolynomialFeatures. We start with the imports: .. code:: python - + import pandas as pd import numpy as np from sklearn.model_selection import train_test_split @@ -194,6 +261,10 @@ Scikit-Learn's PolynomialFeatures. from feature_engine.encoding import OrdinalEncoder from feature_engine.wrappers import SklearnTransformerWrapper +Next, we load the Titanic dataset and split it into a train set and a test set: + +.. code:: python + X, y = load_titanic( return_X_y_frame=True, predictors_only=True, @@ -202,6 +273,12 @@ Scikit-Learn's PolynomialFeatures. X_train, X_test, y_train, y_test= train_test_split(X, y, test_size=0.2, random_state=42) +Now, we assemble the pipeline. The last step wraps scikit-learn's PolynomialFeatures with +:class:`SklearnTransformerWrapper()`, so that it only creates interactions between +`pclass` and `sex`: + +.. code:: python + pipeline = Pipeline(steps = [ ('ci', CategoricalImputer(imputation_method='frequent')), ('mmi', MeanMedianImputer(imputation_method='mean')), @@ -211,12 +288,18 @@ Scikit-Learn's PolynomialFeatures. variables=['pclass','sex'])) ]) +Finally, we fit the pipeline and use it to transform the train and test sets: + +.. code:: python + pipeline.fit(X_train) X_train_transformed = pipeline.transform(X_train) X_test_transformed = pipeline.transform(X_test) print(X_train_transformed.head()) +We see the resulting dataframe, with the new interaction features at the end: + .. code:: python age sibsp parch fare cabin embarked pclass sex \ @@ -239,7 +322,7 @@ More details In the following Jupyter notebooks you can find more details about how to navigate the parameters of the :class:`SklearnTransformerWrapper()` and also access the parameters -of the Scikit-learn transformer wrapped, as well as the output of the transformations. +of the scikit-learn transformer wrapped, as well as the output of the transformations. - `Wrap sklearn categorical encoder `_ - `Wrap sklearn KBinsDiscretizer `_ diff --git a/docs/user_guide/wrappers/index.rst b/docs/user_guide/wrappers/index.rst index af07bef25..afedad42b 100644 --- a/docs/user_guide/wrappers/index.rst +++ b/docs/user_guide/wrappers/index.rst @@ -3,7 +3,7 @@ Scikit-learn Wrapper ==================== -Feature-engine's Scikit-learn wrappers wrap Scikit-learn transformers allowing their +Feature-engine's scikit-learn wrappers wrap scikit-learn transformers allowing their implementation only on a selected subset of features. .. toctree:: diff --git a/docs/versions/index.rst b/docs/versions/index.rst index dc33a6a62..6ec3ce2db 100644 --- a/docs/versions/index.rst +++ b/docs/versions/index.rst @@ -3,5 +3,5 @@ Other versions Web-based documentation is available for versions listed below: -- `Feature-engine 1.6 `_ +- `feature-engine 1.6 `_