jitter control loop for predictive backpressure scheduling

[GGraziadei]

What is the purpose of the change

ToDo

How was the change tested

ToDo

In the context of #8583

[reiabreu]

@GGraziadei can this one be closed?

[GGraziadei]

Hi @reiabreu

I am currently working asynchronously to propose a code change aimed at stabilizing jitter without impacting throughput or increasing latency.

Unlike the discrete event simulation results, the feedback tuple being evaluated with storm perf is currently degrading performance.

To address this, I am trying to create a feedback tick stream to replace the current ratio-based mechanism.

If performance remains degraded even after this modification, I will post an update and either close the PR or ask for further help in Dev maillist.

[GGraziadei]

Hi @reiabreu, I can confirm that the progress on this PR hasn't been lost.

I successfully reproduced these results locally using the storm-perf/FileReadWordCountTopo on a storm development cluster. I am currently refining the solution and will commit the changes to convert this draft into a formal PR soon.

Apologies for the delay—it took some time to verify, but I am confident that using the FeedbackStream tick is the right approach to propagate the control signal without impacting too much on throughput or increasing latency.

Metric	No FeedbackTuple (Reference)	FeedbackTuple	Variance
Elapsed Time (sec)	120	120	0.00%
Workers	2	2	0.00%
Tasks	7	7	0.00%
Executors	7	7	0.00%
Transferred (messages)	35,338,000	34,518,000	-2.32%
Transfer Rate (messages/s)	588,966	575,300	-2.32%
Spout Executors	1	1	0.00%
Spout Transferred (messages)	4,444,000	4,388,000	-1.26%
Spout Acks	4,412,000	4,298,000	-2.58%
Spout Throughput (acks/s)	74,066	73,133	-1.26%
Spout Avg Complete Latency (ms)	457.837	467.308	+2.07%
Spout Max Complete Latency (ms)	457.837	467.308	+2.07%