Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUGFIX] Enable SlackNotificationAction variable substitution #10443

Merged
merged 11 commits into from
Oct 2, 2024

Conversation

cdkini
Copy link
Member

@cdkini cdkini commented Sep 30, 2024

We should allow for ${VAR} syntax with this action

  • Description of PR changes above includes a link to an existing GitHub issue
  • PR title is prefixed with one of: [BUGFIX], [FEATURE], [DOCS], [MAINTENANCE], [CONTRIB]
  • Code is linted - run invoke lint (uses ruff format + ruff check)
  • Appropriate tests and docs have been updated

For more information about contributing, see Contribute.

After you submit your PR, keep the page open and monitor the statuses of the various checks made by our continuous integration process at the bottom of the page. Please fix any issues that come up and reach out on Slack if you need help. Thanks for contributing!

Copy link

netlify bot commented Sep 30, 2024

Deploy Preview for niobium-lead-7998 ready!

Name Link
🔨 Latest commit 1a7b959
🔍 Latest deploy log https://app.netlify.com/sites/niobium-lead-7998/deploys/66fd9dd3cee72200081d5ffb
😎 Deploy Preview https://deploy-preview-10443.docs.greatexpectations.io
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

Copy link

codecov bot commented Sep 30, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 80.09%. Comparing base (09bdbcc) to head (1a7b959).
Report is 2 commits behind head on develop.

✅ All tests successful. No failed tests found.

Additional details and impacted files
@@           Coverage Diff            @@
##           develop   #10443   +/-   ##
========================================
  Coverage    80.08%   80.09%           
========================================
  Files          461      461           
  Lines        39979    39990   +11     
========================================
+ Hits         32019    32030   +11     
  Misses        7960     7960           
Flag Coverage Δ
3.10 67.72% <100.00%> (+0.01%) ⬆️
3.11 67.72% <100.00%> (+0.01%) ⬆️
3.12 66.34% <100.00%> (+0.03%) ⬆️
3.12 aws_deps 45.94% <41.17%> (-0.01%) ⬇️
3.12 big 54.58% <41.17%> (-0.01%) ⬇️
3.12 filesystem 61.40% <82.35%> (+0.04%) ⬆️
3.12 mssql 50.05% <41.17%> (-0.01%) ⬇️
3.12 mysql 50.11% <41.17%> (-0.01%) ⬇️
3.12 postgresql 54.41% <41.17%> (+0.03%) ⬆️
3.12 spark 57.90% <41.17%> (-0.01%) ⬇️
3.12 spark_connect 46.25% <41.17%> (+<0.01%) ⬆️
3.12 trino 52.51% <41.17%> (+<0.01%) ⬆️
3.8 67.75% <100.00%> (+<0.01%) ⬆️
3.8 athena or clickhouse or openpyxl or pyarrow or project or sqlite or aws_creds 55.22% <41.17%> (+0.03%) ⬆️
3.8 aws_deps 45.97% <41.17%> (-0.01%) ⬇️
3.8 big 54.60% <41.17%> (-0.01%) ⬇️
3.8 databricks 47.72% <41.17%> (+0.04%) ⬆️
3.8 filesystem 61.42% <82.35%> (+0.04%) ⬆️
3.8 mssql 50.04% <41.17%> (-0.01%) ⬇️
3.8 mysql 50.10% <41.17%> (-0.01%) ⬇️
3.8 postgresql 54.40% <41.17%> (+0.03%) ⬆️
3.8 snowflake 48.57% <41.17%> (+0.04%) ⬆️
3.8 spark 57.87% <41.17%> (-0.01%) ⬇️
3.8 spark_connect 46.26% <41.17%> (+<0.01%) ⬆️
3.8 trino 52.50% <41.17%> (+<0.01%) ⬆️
3.9 67.73% <100.00%> (+0.01%) ⬆️
cloud 0.00% <0.00%> (ø)
docs-basic 52.58% <94.11%> (+0.01%) ⬆️
docs-creds-needed 52.81% <94.11%> (+0.01%) ⬆️
docs-spark 52.24% <94.11%> (+0.03%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

slack_channel: Optional[str] = None
slack_webhook: Optional[Union[ConfigStr, str]] = None
slack_token: Optional[Union[ConfigStr, str]] = None
slack_channel: Optional[Union[ConfigStr, str]] = None
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should a channel be a secret str? Felt less likely than the rest but I think consistency makes sense here

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would go ahead and keep it substituable, although not because it needs to be secret.
It adds flexibility without further complicating anything.

slack_webhook = values["slack_webhook"]
from great_expectations.data_context.data_context.context_factory import project_manager

config_provider = project_manager.get_config_provider()
Copy link
Contributor

@Kilo59 Kilo59 Sep 30, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

non-blocking

Just thinking out loud... but now that we can pull the config provider from the project_manager we don't actually need to pass it to ConfiStr.get_config_value(config_provider).

We could always have this method ConfiStr.get_config_value() pull from the project manager.

def get_config_value(self, config_provider: _ConfigurationProvider) -> str:
"""
Resolve the config template string to its string value according to the passed
_ConfigurationProvider.
"""
LOGGER.info(f"Substituting '{self}'")
return config_provider.substitute_config(self.template_str)

I would do this separately from this PR.

@cdkini cdkini enabled auto-merge October 2, 2024 15:52
@cdkini cdkini added this pull request to the merge queue Oct 2, 2024
Merged via the queue into develop with commit 70a651a Oct 2, 2024
69 checks passed
@cdkini cdkini deleted the m/core-455/slack_action_substitution branch October 2, 2024 20:17
@leodrivera
Copy link
Contributor

Hello @cdkini 

I've tested your implementation, and it seems to be working fine. However, the checkpoint json files are being saved with the values substituted, not the placeholders. As this file is intended to be committed alongside the other gx config files, I believe this is not an intended behavior.

I think that the expected behavior should just be like the connection_string from the add_or_update_postgres method, where the placeholders are saved in the great_expectations.yml file.

Comment on lines +244 to +247
values[credential] = cls._substitute_slack_credential(
slack_credential=values[credential],
config_provider=config_provider,
)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@cdkini sorry I missed this in the review.
The values shouldn't be updated in the validator method. They should only be substituted when they are actually needed which looks like that would be inside the .run() method.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants