Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Improve] Make DeserializationSchema.deserialize return List #5833

Open
wants to merge 1 commit into
base: dev
Choose a base branch
from

Conversation

ruanwenjun
Copy link
Member

@ruanwenjun ruanwenjun commented Nov 12, 2023

Purpose of this pull request

Right now, if we want to use DeserializationSchema, we will need to provide a Collector, this make it not convenient use in the whole project.

And sometimes we want to set tableId to SeatunnelRow after deserialize e.g. in multipletable mode, so it's better to make DeserializationSchema be a real util only return SeaTunnel rows.

Does this PR introduce any user-facing change?

How was this patch tested?

Check list

@ruanwenjun ruanwenjun force-pushed the dev_wenjun_refactorDeserializationSchema branch 16 times, most recently from 9b34eaf to ddaf6f6 Compare November 16, 2023 07:15
@ruanwenjun ruanwenjun force-pushed the dev_wenjun_refactorDeserializationSchema branch from ddaf6f6 to 8193c00 Compare November 24, 2023 03:21
Comment on lines 47 to 51
if (seaTunnelRows.size() != 1) {
log.warn(
"The AmazonSqsDeserializer only support one row, but got {} rows, will drop the extra rows",
seaTunnelRows.size());
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think throw exception would be better.

@ruanwenjun ruanwenjun force-pushed the dev_wenjun_refactorDeserializationSchema branch from 8193c00 to 55ad788 Compare November 27, 2023 11:32
@ruanwenjun ruanwenjun force-pushed the dev_wenjun_refactorDeserializationSchema branch from 55ad788 to 870100f Compare November 28, 2023 02:34
@hailin0
Copy link
Member

hailin0 commented Dec 1, 2023

check ci

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants