Create a comparison of imputation strategies #86

anilkumarpanda · 2021-03-04T13:16:01Z

Initial draft of the imputation strategy comparison . Resolves #73
Adds the following feature to Probatus :

Given a set of imputation strategies, show which one performs the best for the given classifier.
For categorical variables, the missing values are filled with 'missing' and a missing indicator is added.
For numerical values, various imputation strategies are applied.
The output is provided in the form of an bar plot as well as report dataframe with the results. e.g.
For models that support handling of missing values by default Model Imputation result is displayed. For models that do not support handling of missing values. Only the strategy results are displayed.

anilkumarpanda · 2021-03-04T15:47:38Z

@Matgrb can you please review this.

Matgrb

Great progress! I left some comments, most improvements can be done on the code side and documentation.

docs/tutorials/nb_imputation_comparison.ipynb

probatus/missing/imputation.py

tests/missing/test_imputation.py

Matgrb

Looks fantastic! I left couple of minor comments, as soon as they are closed we can merge and make a release!

probatus/missing_values/imputation.py

docs/tutorials/nb_imputation_comparison.ipynb

Matgrb

Great work! Very high quality contribution!

anilkumarpanda added 9 commits February 25, 2021 15:46

Initial code creation.

e553b6a

Intial setup for mvi comparison.

120d6e4

Added handling of categorical variables.

b51501f

Bare bone implementation

964f3fa

Added support for RF,LR models in a elegant way.

57edb3f

Added notebook and cleaned up tests.

43da42a

Resolving pyflakes and code for pipelines.

be1e54e

Skipping LGBM tests

d7d1ff3

Skipping LGBM tests

802a41b

Matgrb suggested changes Mar 4, 2021

View reviewed changes

anilkumarpanda added 7 commits March 10, 2021 23:38

Renaming class and adjusting notebooks.

eb21206

Resolving MR comments. Simplifying code by removing parameters.

d33abbf

Updating tests.

396496f

Removed unwanted imports

3c2a5ed

Updated example, removed lgbm.

3ab2596

Skipping lgbm tests

192f32c

Adding lgb tests

f86aeba

Matgrb suggested changes Mar 11, 2021

View reviewed changes

Matgrb reviewed Mar 11, 2021

View reviewed changes

docs/tutorials/nb_imputation_comparison.ipynb Show resolved Hide resolved

anilkumarpanda added 8 commits March 13, 2021 01:33

Resolving mr issues

d616fcd

Updated the plots toshow cv train results as well.

91d9389

Running black on tests and removing ligm tests.

8b73b8d

Adding lgbm tests back

5b9a739

Minor updates to code example in docstring

3b01af8

Updated lgbm test

d45b1cf

Removed requirement.txt

e1a210b

Adding new line to eof

5064ce1

Matgrb approved these changes Mar 15, 2021

View reviewed changes

Matgrb merged commit 97b0661 into ing-bank:main Mar 15, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Create a comparison of imputation strategies #86

Create a comparison of imputation strategies #86

anilkumarpanda commented Mar 4, 2021 •

edited

Loading

anilkumarpanda commented Mar 4, 2021

Matgrb left a comment

Matgrb left a comment

Matgrb left a comment

Create a comparison of imputation strategies #86

Create a comparison of imputation strategies #86

Conversation

anilkumarpanda commented Mar 4, 2021 • edited Loading

anilkumarpanda commented Mar 4, 2021

Matgrb left a comment

Choose a reason for hiding this comment

Matgrb left a comment

Choose a reason for hiding this comment

Matgrb left a comment

Choose a reason for hiding this comment

anilkumarpanda commented Mar 4, 2021 •

edited

Loading