Flink Tumbling Window labelling The Next CEO of Stack OverflowFlink Window and State MaintenanceApache Flink: calculating and outputting multiple values in one windowflink streaming window triggerHow to properly initialize task state at Apache Flink?Flink Kafka consumer StreamExecutionEnvironment only?How to join three or more datastreams /tables on a given key and a common window by datastrem API or Flink Table API/SQL?Flink performance issue with sliding time windowFlink windowing: aggregate and output to sinkFlink Sliding count window behaviorFlink Side output for Sliding time window
Is it professional to write unrelated content in an almost-empty email?
What happens if you roll doubles 3 times then land on "Go to jail?"
Sending manuscript to multiple publishers
What's the best way to handle refactoring a big file?
MessageLevel in QGIS3
Multiple labels for a single equation
To not tell, not take, and not want
What is "(CFMCC)" on an ILS approach chart?
Inappropriate reference requests from Journal reviewers
If the heap is initialized for security, then why is the stack uninitialized?
Why don't programming languages automatically manage the synchronous/asynchronous problem?
Is micro rebar a better way to reinforce concrete than rebar?
Return the Closest Prime Number
Can I run my washing machine drain line into a condensate pump so it drains better?
Complex fractions
WOW air has ceased operation, can I get my tickets refunded?
How to count occurrences of text in a file?
Is there a difference between "Fahrstuhl" and "Aufzug"
Why do remote companies require working in the US?
How do I reset passwords on multiple websites easily?
How to start emacs in "nothing" mode (`fundamental-mode`)
What happened in Rome, when the western empire "fell"?
Should I tutor a student who I know has cheated on their homework?
Limits on contract work without pre-agreed price/contract (UK)
Flink Tumbling Window labelling
The Next CEO of Stack OverflowFlink Window and State MaintenanceApache Flink: calculating and outputting multiple values in one windowflink streaming window triggerHow to properly initialize task state at Apache Flink?Flink Kafka consumer StreamExecutionEnvironment only?How to join three or more datastreams /tables on a given key and a common window by datastrem API or Flink Table API/SQL?Flink performance issue with sliding time windowFlink windowing: aggregate and output to sinkFlink Sliding count window behaviorFlink Side output for Sliding time window
I have a scenario with a flink application that receives data streams in the following format:
"event_id": "c1s2s34", "event_create_timestamp": "2019-03-07 11:11:23", "amount": "104.67"
I am using the following tumbling window to find the sum, count, and average amounts for input streams in the last 60 seconds.
keyValue.timeWindow(Time.seconds(60))
However how can I label the aggregated outcome such that I can say that the output data stream between 16:20 and 16:21 the aggregated results are sum x, count y, and average z.
Any help is appropriated.
apache-flink data-stream
asked Mar 7 at 15:25
observer0107observer0107
1
add a comment |
I have a scenario with a flink application that receives data streams in the following format:
"event_id": "c1s2s34", "event_create_timestamp": "2019-03-07 11:11:23", "amount": "104.67"
I am using the following tumbling window to find the sum, count, and average amounts for input streams in the last 60 seconds.
keyValue.timeWindow(Time.seconds(60))
However how can I label the aggregated outcome such that I can say that the output data stream between 16:20 and 16:21 the aggregated results are sum x, count y, and average z.
Any help is appropriated.
apache-flink data-stream
asked Mar 7 at 15:25
observer0107observer0107
1
How do you want to consume the results -- are you going to print them, or write them to a file, or send them to Kafka, ... ?
– David Anderson
Mar 7 at 17:38
Hi David, I want to send the result to Kinesis Firehose.
– observer0107
Mar 11 at 20:31
add a comment |
I have a scenario with a flink application that receives data streams in the following format:
"event_id": "c1s2s34", "event_create_timestamp": "2019-03-07 11:11:23", "amount": "104.67"
I am using the following tumbling window to find the sum, count, and average amounts for input streams in the last 60 seconds.
keyValue.timeWindow(Time.seconds(60))
However how can I label the aggregated outcome such that I can say that the output data stream between 16:20 and 16:21 the aggregated results are sum x, count y, and average z.
Any help is appropriated.
apache-flink data-stream
asked Mar 7 at 15:25
observer0107observer0107
1
I have a scenario with a flink application that receives data streams in the following format:
"event_id": "c1s2s34", "event_create_timestamp": "2019-03-07 11:11:23", "amount": "104.67"
I am using the following tumbling window to find the sum, count, and average amounts for input streams in the last 60 seconds.
keyValue.timeWindow(Time.seconds(60))
However how can I label the aggregated outcome such that I can say that the output data stream between 16:20 and 16:21 the aggregated results are sum x, count y, and average z.
Any help is appropriated.
apache-flink data-stream
apache-flink data-stream
asked Mar 7 at 15:25
observer0107observer0107
1
asked Mar 7 at 15:25
observer0107observer0107
1
asked Mar 7 at 15:25
observer0107observer0107
1
asked Mar 7 at 15:25
observer0107observer0107
1
asked Mar 7 at 15:25
observer0107observer0107
1
1
How do you want to consume the results -- are you going to print them, or write them to a file, or send them to Kafka, ... ?
– David Anderson
Mar 7 at 17:38
Hi David, I want to send the result to Kinesis Firehose.
– observer0107
Mar 11 at 20:31
add a comment |
How do you want to consume the results -- are you going to print them, or write them to a file, or send them to Kafka, ... ?
– David Anderson
Mar 7 at 17:38
Hi David, I want to send the result to Kinesis Firehose.
– observer0107
Mar 11 at 20:31
How do you want to consume the results -- are you going to print them, or write them to a file, or send them to Kafka, ... ?
– David Anderson
Mar 7 at 17:38
How do you want to consume the results -- are you going to print them, or write them to a file, or send them to Kafka, ... ?
– David Anderson
Mar 7 at 17:38
Hi David, I want to send the result to Kinesis Firehose.
– observer0107
Mar 11 at 20:31
Hi David, I want to send the result to Kinesis Firehose.
– observer0107
Mar 11 at 20:31
add a comment |
1 Answer
1
active
oldest
votes
If you look at the windowing example in the Flink training site -- https://training.ververica.com/exercises/hourlyTips.html -- you'll see an example of how to use a ProcessWindowFunction to create output events from windows that include the timing information, etc. The basic idea is that the process() method on a ProcessWindowFunction is passed a Context which in turn contains the Window object, from which you can determine the start and ending times for the window, e.g, context.window().getEnd().
You can then arrange for your ProcessWindowFunction to return Tuples or POJOs that contain all of the information you want to include in your reports.
answered Mar 7 at 17:50
David AndersonDavid Anderson
6,60421424
add a comment |
Your Answer
StackExchange.ifUsing("editor", function ()
StackExchange.using("externalEditor", function ()
StackExchange.using("snippets", function ()
StackExchange.snippets.init();
);
);
, "code-snippets");
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "1"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f55047304%2fflink-tumbling-window-labelling%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
If you look at the windowing example in the Flink training site -- https://training.ververica.com/exercises/hourlyTips.html -- you'll see an example of how to use a ProcessWindowFunction to create output events from windows that include the timing information, etc. The basic idea is that the process() method on a ProcessWindowFunction is passed a Context which in turn contains the Window object, from which you can determine the start and ending times for the window, e.g, context.window().getEnd().
You can then arrange for your ProcessWindowFunction to return Tuples or POJOs that contain all of the information you want to include in your reports.
answered Mar 7 at 17:50
David AndersonDavid Anderson
6,60421424
add a comment |
If you look at the windowing example in the Flink training site -- https://training.ververica.com/exercises/hourlyTips.html -- you'll see an example of how to use a ProcessWindowFunction to create output events from windows that include the timing information, etc. The basic idea is that the process() method on a ProcessWindowFunction is passed a Context which in turn contains the Window object, from which you can determine the start and ending times for the window, e.g, context.window().getEnd().
You can then arrange for your ProcessWindowFunction to return Tuples or POJOs that contain all of the information you want to include in your reports.
answered Mar 7 at 17:50
David AndersonDavid Anderson
6,60421424
add a comment |
If you look at the windowing example in the Flink training site -- https://training.ververica.com/exercises/hourlyTips.html -- you'll see an example of how to use a ProcessWindowFunction to create output events from windows that include the timing information, etc. The basic idea is that the process() method on a ProcessWindowFunction is passed a Context which in turn contains the Window object, from which you can determine the start and ending times for the window, e.g, context.window().getEnd().
You can then arrange for your ProcessWindowFunction to return Tuples or POJOs that contain all of the information you want to include in your reports.
answered Mar 7 at 17:50
David AndersonDavid Anderson
6,60421424
If you look at the windowing example in the Flink training site -- https://training.ververica.com/exercises/hourlyTips.html -- you'll see an example of how to use a ProcessWindowFunction to create output events from windows that include the timing information, etc. The basic idea is that the process() method on a ProcessWindowFunction is passed a Context which in turn contains the Window object, from which you can determine the start and ending times for the window, e.g, context.window().getEnd().
You can then arrange for your ProcessWindowFunction to return Tuples or POJOs that contain all of the information you want to include in your reports.
answered Mar 7 at 17:50
David AndersonDavid Anderson
6,60421424
answered Mar 7 at 17:50
David AndersonDavid Anderson
6,60421424
answered Mar 7 at 17:50
David AndersonDavid Anderson
6,60421424
answered Mar 7 at 17:50
David AndersonDavid Anderson
6,60421424
6,60421424
add a comment |
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f55047304%2fflink-tumbling-window-labelling%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
How do you want to consume the results -- are you going to print them, or write them to a file, or send them to Kafka, ... ?
– David Anderson
Mar 7 at 17:38
Hi David, I want to send the result to Kinesis Firehose.
– observer0107
Mar 11 at 20:31