MNS Report1
MNS Report1
Tiruchirappalli
(NIT-Trichy)
Submitted by
[Link] (406124054)
1
A Network tool assisted TCP Packet Loss Prediction Using Machine Learning
Abstract TCP Congestion control in real time dataset is important to improve the
performances and Quality of Service. There are many proactive congestion control
algorithms are there. Here the reactive congestion control Reno algorithm is used along with
heuristic machine learning approach. The random forest is works better in packet loss
prediction with classifier and cross validation. The backoff sender is used with Explicit
congestion notification enabled at congestion experienced bit at TCP header to inform about
the congestion.
1. Introduction
To avoid the packet loss due to congestion in transmission control protocol in transport layer
is important to improve the throughput and to avoid packet retransmission. The reno method
is previously used to control the congestion. The reno works on the basis of additive increase
and multiplicative decrease (AIMD) strategy. The active queue congestion window size
(cwnd) is increased by one after each round-trip time (RTT) which means from the request of
packet, sending the packets and reception of packet time but the usual way is increasing the
packet size after receiving the acknowledgement from the receiver. The additive increase
concept is raising the cwnd queue size till packet loss event occurs or duplicate
acknowledgement and multiplicative decrease is decreasing half size of cwnd this AIMD.
Then reno will go into fast recovery which means after half decrease the window gets
increased additively when the initial lost packet is received or time out occurs. After that it
goes into slow start phase which is cwnd size set to 1. This is robust and it is work in any
network environment. There is high bandwidth algorithms are there compared but reno is
underutilization of bandwidth. Another TCP congestion control mechanism is cubic which
apply cubic function in increasing the cwnd. The window size will be increased based on
concave and convex manner. In concave increase the window get increased quickly and
reached the last congestion window size. In convex type at grow slowly and fast at the end.
This would not depend on RTT or acknowledgement-based increase. This is based on last
cwnd decreased state like based on real time data bandwidth. This reno and cubic are reactive
protocol which will increase the performance of data transmission.
2
So, after packet loss, taking action is reactive but need to proactive that is before
packet loss occurs need to decrease the cwnd and get into congestion control mechanism.
This will reduce the packet loss as well as reducing the packet retransmission. In Bottleneck
Bandwidth, retransmission timeout and round trip time propagation (BBR) [3] they will do
the congestion control analyzing the bandwidth bottle neck and modifying the transmission
rate by considering prediction. This will increase the amount of packets send per given time
that is throughput and reduce packet lost which reduces the packet queue pressure. This
works based on the estimation of bandwidth bottleneck by taking the delivery rate
measurement and RTT. The advantage is high throughput, lower latency and it works better
in high-speed network. But in short RTT and its queue build causes the problem in predicting
the bottleneck. In Low Extra Delay Background Transport (LEDBT) [4] took the one-way
delay if the reception of packets at increasing time count which shows the queue size and do
measurement band dynamically tuning the congestion window size. This will reach high
bandwidth and adjusting the congestion active window size before the data get loss occurs.
Network Assisted Dynamic Adaptation (NADA) [7] addressed the congestion control in
interactive media application like video conferencing which is susceptible to dynamic
bandwidth and propagation delay. In Explicit Congestion Notification (ECN) [9] the
congestion in network packet is marked by the receiver and informed to sender by set the bit
in TCP data packet header. The expilicit bit notification can also be done by the router also.
The sixth and seventh bit 2 bits are dedicated for ECN. sixth bit is ECT ECN - present
transport bit and congestion known. If both bits value is 00 then ECN is not followed. The
bits 01 and 10 set by sender, 11 send by the router to sender to indicate the congestion.
The objective is for real time application the bandwidth is not considered and the
robustness highly matters, so reno algorithm is used to generate the dataset. since it is the
reactive algorithm planned to incorporate the machine learning prediction algorithm to reduce
the cwnd without packet loss. The ECN technique is used for congestion control by the
receiver intimation to sender by enabling 2 bits and other parameters also. Then 01 and 10 is
used by receiver for ECN technique used and 00 not used. The value 11 used to set ECN bits
by router.
2. Literature Review
2.1. Real-Time TCP Packet Loss Prediction Using Machine Learning, IEEE Access journal
2024. The objective is in view of replacing the heuristics machine learning approach TCP
packet loss prediction algorithms based on delay and/or throughput, doing reaction when
3
packet loss is predicted to occur. The techniques used is Forward Erasure Correction (FEC) is
used to protect the packet loss and also dynamically tuning the parameters. Reno and cubic
flows with XGBoost ML model used. They used mininet to capture the dataset. The test
result is Trained and fined tuned in emulation test reno and cubic connections over a single
bottleneck. ECN is used to reno cubic backoff timer before packet loss occurs. The drawback
is ECN will decrease the transmission count based on data loss prediction to avoid or reduce
the packet retransmission.2.2. A machine learning approach for packet loss prediction in
science flows, Future generation computer systems, 2020 In scientific data transfers will
affect the data presence in active queue management. Large data transfer cause network
performance degradation, delay in queue, dropping packets. It is a tool to predict end to end
retransmission in an arbitrary size of data flow. The limited set of required path and node
measurements are required mandatory in input attributes. Random forest algorithm with short
training time is used to predict the retransmission rate. Able to correlate different features in
packet loss and data retransmission. The drawback is performance degradation in terms of
throughput is not addressed.
2.3. Predictive modeling of loss ratio for congestion control in IoT networks using deep
learning,’’ in Proc. IEEE Global Commun. Conf., Dec. 2023, pp. 2814–2819.
Congestion occurs when flow of packets in many directions share the same networks. This
problem is increased by low power lossy networks which leads to latency, reduced
throughput, packet loss. To provide the improved Quality of service the ratio of packet loss
from packet transmitted. This is the important metric to to analyse the network traffic load
and congestion level.
2.4. Congestion or No Congestion: Packet Loss Identification and Prediction Using Machine
Learning 2024
Many factors are the reason for packet loss exceot congestion. In wireless networks the
packet loss is also due to communication like issues. The Tcp should react the same way as
like congestion when there is no congestion is reason for packet loss. Random forest and
KNN algorithms are used to predict packet loss and to improve network performance. The
features like jitter, RTT and cwnd are important to rely on network performance. The random
forest enhances the performance in large subset of unique decision trees is built from random
subset of data and attributes to familiarize the diversity in training and reduces the risk of
overfitting. These findings suggest that integrating these models into live networks could
4
significantly, enhance network performance. So focus on refining these models for practical
deployment in dynamic network environments.
2.5. Can We Exploit Machine Learning to Predict Congestion over mmWave 5G Channels,
Journal
Here congestion state like taking large time in delay and packet arrival time among the node
metrics of TCP congestion are analyzed. K means work well in binary classification but for
high bandwidth In hierarchical clustering based on k means approach is suitable for multi-
level classification. But the issues not addressed are different metrics with various
combinations are tried to improve the network performance. Also find the proportional
relationship between transport layer metrics. Need to test in heterogeneous deployments like
LTE and 5G networks. The paper finds a clear correlation between the moving standard
deviation of the delay and congestion, but only a weak correlation between the moving
average of the delay and congestion. These findings could be informative for an ML feature
selection and construction process such as ours, although 5G mm Wave connections are not a
part of our considered dataset.
2.6. End-to-end congestion control approaches for high throughput and low delay in 4G/5G
cellular networks 2021
The cellular network support high bit rates and low loss communication. But the application
and services running gets into congestion challenging phase. To handle this congestion
control algorithms CCA are designed. Three approaches are followed they are reactive,
predictive and network assisted CCA. In predictive algorithm fixed interval, dynamic interval
methods used. In reactive Algorithms loss triggered, delay triggered, rate triggered and hybrid
metods are used. In network assisted algorithms in band signaling, out of band signaling
concepts are used. In predictive environment signals are used to handle the impact of wrong
predictions. Here the objective is to improve the throughput and reduce latency. But it fails to
adapt the degree of variability in data and interaction among CCA shared queueing bottleneck
technique.
2.7. Improving Internet Congestion Control with Packet Loss Prediction Using Machine
Learning 2023
Bandwidth Bottleneck and round-trip time, Low extra delay background transport algorithms
are used. XGBoost and LightGBM Machine learning algorihms are used. Reno and cubic
5
classifier used. predict packet loss and proactively act against congestion. If conjestion occurs
Reduce the sending rate. They use mininet to generate the data from data traffic in real time.
But it didn't address the FEC to dynamically decide when to add redundancy to the data at the
sender-based model predictions.
The objective is to node to node congestion control in computer networks — a core function
for ensuring efficient and fair data transmission. The methodology used is Remy,
performance-oriented congestion control,aurora. They considered the following timer_name,
expire_time, retrans, rto, rtt, rtt_variance, min_rtt, max_rtt, cwnd, cwnd_diff, min_cwnd,
max_cwind, ssthresh, min_thresh, max_thresh, data_segments_sent, last_send, pacing_rate
input metrics, reinforcement learning is used. The drawback is Training environment
mismatch, stability and fairness,interpretability
Algorithm & Year Simulator/
Objective Methodology Input Metrics for making decision Requirements/Assumptions Achieved enhancements Limitations If any thing
of publication Machine learning concept
In view of replacing the heuristics machine timer_name,expire_time,retrans,rto,rtt,rtt Dataset are created
On sender side polling using ss
learning approach TCP congestion control Forward Erasure Correction (FEC) is _variance,min_rtt,max_rtt,cwnd,cwnd_d using mininet, such
Reno and cubic, and ran TSHARK in capture mode Reduce the sedning rate proactively
algorithms based on delay and/or used to protect the packet loss and also iff,min_cwnd,max_cwind,ssthresh, XGBoost controlled environment -
XGBoost, 2024 to find the incomming and to reduce the data retransmission
throughput, doing the reaction when packet dynamically tuning the parameters. min_thresh,max_thresh,data_segments_s does not create the
outgoing packets
loss is predicted to occur. ent,last_send,pacing_rate varible and complex
Packet loss in scientific data transfers will Solution to predict packet retransmission
To smooth the data automatic
affect the data availability. Large data in scientific data transmission or Prediction of retransmission
Random forest, tcp_rtt_avg,tcp_initial_cwin,tcp_win_m smoothing for attention Throughtput not
transfer cause network performance arbitrary size data. The accuracy is good percentage using different Random forest -
2020 ax priortization in time series addressed
degradation, delay in queue, dropping with variability in feature distribution combination of variables.
algorithm is used
packets. among various data set. To avoid noise
Deep Learning, Reliable data transfer is necessary in health DL technique to predict reveived signal DL TCP with loss based, delay It captured intricate temporal Basic RNN, Advanced RNN and Significanct lag in
Time series data,observed loss ratio -
2023 monitoring sysrtem using IoT strength and packet delievery ratio basec and hybrid based dependencies within different CNN network performance
Congestion is not the only reason for packet Due to the high imbalance in our
Refining these models
loss. In wireless networks the packet loss The Tcp should react the same way as dataset, metrics like precision and Individual decision trees is built
Random forest and Total packets, total drop packets, for practical
is also due to communication like issues. like congestion when there is no accuracy are skewed by the from random subset of data and Random forest -
KNN, 2024 congestive drop, non congestive drop deployment in dynamic
The Tcp should react the same way as like congestion is reason for packet loss. predominance of the majority attributes to familiarize the diversity
network environments.
congestion when there is no congestion is class.
Here congestion state like delay and inter
Different metrics with
arrival time metrics of TCP congestion are These findings could be informative
The paper finds a clear correlation various combinations
analyzed. K means work well in binary for an ML feature selection and
between the moving standard deviation K means with multi level are tried to improve the
classification but for high data rates. In construction process such as ours,
Kmeans, 2020 of the delay and congestion, but only a Threshold,cwnd,RTT classification with hierarchical Hierarchical k means clustering network performance. -
hierarchical clustering based on k means although 5G mmWave connections
weak correlation between the moving solution, HMM, RNN Also find the linear
approach is suitable for multi level are not a part of our considered
average of the delay and congestion. relationship between
classification. But the issues not addressed dataset.
transport layer metrics.
are different metrics with various
Link adaptation, mmwave, small cells, Handle degree of
Congestion control To increase the thrighput and reduce dual connectivity, carrier aggregation, Multi layer solution, tracking the Predictive, reactive and network
variation in data and
End to end congestion control algorithms - -
algorithms, 2021 latency MIMO, dynamic network slicing, and highly variable bandwidth assited algorithms communication among
resources allocation, converged shared queueing
XGBoost,LightGB Predict packet loss and proactively act Explicit congestion notification, scale pos weight, Learning FEC to dynamically
Reno and cubic algorithm data set Reduce the sending rate mininet and XGBoost, Light GBM -
M, 2024 against congestion Forward Erasure correction rate,gamma,Max depth,Min child decide when to add
timer_name,expire_time,retrans,rto,rtt,rtt
Training environment
End-to-end congestion control in computer _variance,min_rtt,max_rtt,cwnd,cwnd_d
Remy,performance oriented congestion Adaptability,generalization,perfrona mismatch, stability and
Reinforced learning networks — a core function for ensuring iff,min_cwnd,max_cwind,ssthresh, Dynamic environment data Reinforced learning -
control,aurora ce improved fairness,
efficient and fair data transmission. min_thresh,max_thresh,data_segments_s
interpretability
ent,last_send,pacing_rate
The dataset is created using NS3 simulator and datset is captured using wireshark.
6
System Model:
Node Creation:
21 nodes are created with 1 router, 10 senders, and 10 receivers. The Mobility Model
with All nodes use a Constant Position Mobility Model, meaning they don't move during the
simulation. The Point-to-Point Links: Senders use a 10 Mbps link to the router with a 5 ms
delay. Receivers use a bottleneck 5 Mbps link with a 50 ms delay to the router. Random
Sender-Receiver Pairing: Receivers are assigned to senders using std::shuffle(). Application
Setup: BulkSendHelper is used for sending traffic from the sender to the receiver.
PacketSinkHelper is used on the receiver side to collect the incoming data. PCAP Export:
EnablePcap captures each packet for both sender and receiver nodes. NetAnim: The positions
of nodes are set in the simulation to visualize them in NetAnim. Random colors are assigned
to each sender and receiver node. Flow Monitoring and Output: Flow statistics (e.g., TX/RX
packets, throughput, lost packets) are collected using the FlowMonitor and saved to CSV and
XML files.
7
FIGURE 2: Running NS3 program
8
FIGURE 4: Packets travelling from Router to Destination Node
The main idea is to simulate multiple TCP flows sharing a common path (bottleneck). If
every sender was directly connected to its receiver, there would be no congestion or
competition, which is unrealistic for real networks. With the router in the middle: i) All
senders send their traffic through the router. ii) All receivers receive data from the router. iii)
The router's link to the receivers is intentionally limited (e.g., 5 Mbps, 50ms) to simulate a
shared bottleneck where congestion and TCP behaviour (like Reno's window backoff)
becomes interesting.
In the real world, data doesn't flow directly from sender to receiver — it passes through one
or more routers. This simulation mirrors that by routing all flows through a common node.
When traffic is forced through a single router, it's easier to measure congestion, packet drops,
and TCP behavior under load. It also lets you observe how different TCP flows compete for
shared bandwidth — perfect for packet loss prediction studies.
9
Topology Summary:
Sender1,2,3,...10 sends the data through router and the receiver is reciever1,2,..10. Each
sender has a private link to the router. The router connects to all receivers over bottleneck
links. So all traffic funnels through the router — creating contention and realistic network
pressure.
If you removed the router and connected each sender directly to a receiver:
The packet loss is constant. Possible Reasons for Constant Packet Loss
• 5 Mbps capacity
• 50 ms delay
This means the Bandwidth-Delay Product (BDP) is limited. As more flows share the link,
TCP congestion control kicks in, and flows back off to avoid more losses. Hence, you don’t
always get proportionally more losses with more senders.
10
B) Machine learning model design with evaluation
Here the explanation about what model is selected and discuss about why followed by
training data, feature selection and model training.
A. Model Selection
The congestion control is a time series based problem. Based on RTT and cwnd the
state of congestion going to change. The congestion occurs or not based on packet
loss. Packet loss may be any positive value. But the one packet loss occurs hen
congestion occurs it doesnt matter of 1 or more packet loss. So this problem can be
considered as binary classification problem. For this binary classification problem,
for the tabulated data XGBoost will works better. But this problem dataset is the
unbalanced dataset. To be consider this random forest machine learning algorithm will
woks better compared to many machine learning algorithm.
B. Obtaining Training Data and Feature selection
1) Data Generation
The dataset generated with 20 and 40 nodes using NS3 simulator and this for
observed with 1 minutes and captured using iPerf TCP connection. Here Reno
algorithm is used to generate the dataset. The dataset is generated also with
wireshark simulator which can also be generated iPerf.
2) Feature Selection
In NS3 simulator dataset the transmitting packets, reciving packets difference
shows the packet loss occurs or not. If the packet loss is 0 then there is no
congestion. This will show the throughput. The TxPackets, RxPackets,
Throughput(bps), LostPackets are the attributes. In wireshark simulation dataset
the rtt and cwnd attributes place the important role. The congestion occurs when
either rtt or cwnd any one increases. The cwnd value attributes are supported by
min_cwnd and max_cwnd. The cwnd value close to min_cwnd then there will be
a little chance for congestion. If the value close to max_cwnd then congestion
occurs for sure. The max_cwnd can be considered as the ss_thresh. To avoid
congestion if cwnd value reaches the ss_thresh then reno cubic algorithm used that
the cwnd size is reduced to halves the cwnd size. The cwnd_diff attribute is that
the difference between current cwnd and previous cwnd. If the cwnd_diff
increases posotivlvey then it shows the cwnd windows increases.
11
The TCP packet lost attribute is to denote the packet loss event occurs or not. That
is binary classification. If Packet loss is 0 or flase then not congestion and no
packet loss occurs. If the value is true or 1 then it shows the congestion and packet
loss occurred. Which can learned from cwnd_difference attributes. If that value is
negative the cwnd value is decreased because the transmission lost occurred. But
the data loss true is vary rare to occur. So the dataset becomes unbalanced because
packet loss event occurred is very minimum sometimes it wont. Smote sampling
tehniques used to balance the dataset.
The objective is to avoid packet loss by sender side backoff techniques which can be
notified by ECN. The updation by explicit congestion can be done by analyzing the
12
throughput in bits per second. But in NS3 captured data using Tshark the packet loss is
normal and same for some of the nodes since 20 nodes considered. The time for data
preparation is 0.20 ms. The random forest machine learning takes the training time is 0.027
ms and prediction time is 0.00225 ms. The accuracy, runtime training time and prediction
time is showed in the table.
Random
93.055556 0.026923 0.002249
Forest
The TCP reno is the reactive algorithm it will react for 1 rtt a time. The Explicit congestion
notification is enabled the congestion experienced bit based on rtt which is in the TCP header
causing the backoff that is it halves the congestion window. After backoff the congestion
window reduced bit is enabled.
Feature Importance
cwnd_diff 0.273858
expire_time 0.173713
cwnd 0.170601
pacing_rate 0.169122
rtt 0.109760
13
Feature Importance
data_segments_sen
0.084819
t
ssthresh 0.018126
14
Figure 6. Correlation between Features
15
to reduce sending rates proactively based on model predictions, potentially lowering the
overhead associated with Forward Erasure Correction (FEC) implementations. In future need
to work in dynamic environment with varying active queue size and neural networks.
REFERENCES
[1] E. Blanton, D. V. Paxson, and M. Allman, TCP Congestion Control, Standard RFC 5681,
Sep. 2009. [Online]. Available: [Link] [Link]/info/rfc5681
[2] L. Xu, S. Ha, I. Rhee, V. Goel, and L. Eggert, CUBIC for Fast and Long Distance
Networks, Standard RFC 9438, Aug. 2023. [Online]. Available:
[Link]
[4] S. Shalunov, G. Hazel, J. Iyengar, and M. Kühlewind, Low Extra Delay Background
Transport (LEDBAT), Standard RFC 6817, Dec. 2012. [Online]. Available: [Link]
[Link]/info/rfc6817 [5] Y. Zaki, T. Pötsch, J. Chen, L. Subramanian, and C. Görg,
‘‘Adaptive congestion control for unpredictable cellular networks,’’ in Proc. ACM Conf.
Special Interest Group Data Commun. New York, NY, USA, Aug. 2015, pp. 509–522, doi:
10.1145/2785956.2787498.
[6] V. Arun and H. Balakrishnan, ‘‘Copa: Practical delay-based congestion control for the
Internet,’’ in Proc. 15th USENIX Symp. Networked Syst. Design Implement. (NSDI), Apr.
2018, pp. 329–342. [Online]. Available:
[Link]
[8] I. Johansson and Z. Sarker, Self-Clocked Rate Adaptation for Multimedia, Standard RFC
8298, Dec. 2017. [Online]. Available: [Link] [Link]/info/rfc8298
16
[10] [Link],[Link],[Link], and [Link],ForwardErasure Correction (FEC) Coding and
Congestion Control in Transport, Jul. 2022. [Online]. Available:
[Link]
[11] C. A. Grazia, M. Klapez, and M. Casoni, ‘‘The new TCP modules on the block: A
performance evaluation of TCP pacing and TCP small queues,’’ IEEE Access, vol. 9, pp.
129329–129336, 2021.
[12] R. Adams, ‘‘Active queue management: A survey,’’ IEEE Commun. Surveys Tuts., vol.
15, no. 3, pp. 1425–1476, 3rd Quart., 2013.
[13] B. Briscoe, K. D. Schepper, M. Bagnulo, and G. White, Low Latency, Low Loss, and
Scalable Throughput (L4S) Internet Service: Architecture, Standard RFC 9330, Jan. 2023.
[Online]. Available: [Link] [Link]/info/rfc9330
[14] T. Chen and C. Guestrin, ‘‘XGBoost: A scalable tree boosting system,’’ in Proc. 22nd
ACM SIGKDD Int. Conf. Knowl. Discovery Data Mining, vol. 11, New York, NY, USA,
Aug. 2016, pp. 785–794, doi: 10.1145/2939672.2939785.
[15] A. Mishra, X. Sun, A. Jain, S. Pande, R. Joshi, and B. Leong, ‘‘The great Internet TCP
congestion control census,’’ Proc. ACM Meas. Anal. Comput. Syst., vol. 3, no. 3, pp. 1–24,
Dec. 2019, doi: 10.1145/3366693.
[18] A. Mishra, L. Rastogi, R. Joshi, and B. Leong, ‘‘Keeping an eye on congestion control in
the wild with nebby,’’ in Proc. ACM SIG COMM Conf. New York, NY, USA, Aug. 2024,
pp.136–150, doi: 10.1145/3651890.3672223.
[19] N. Cardwell, I. Swett, and J. Beshay. (Jul. 2024). BBR Congestion Control. Internet Eng.
Task Force. [Online]. Available: [Link]
[20] M. Mathis, J. Semke, J. Mahdavi, and T. Ott, ‘‘The macroscopic behavior of the TCP
congestion avoidance algorithm,’’ ACM SIGCOMM Comput. Commun. Rev., vol. 27, no. 3,
pp. 67–82, Jul. 1997, doi: 10.1145/263932.264023.
17
[21] A. Gurtov, T. Henderson, S. Floyd, and Y. Nishida, The NewReno Modification to TCP’s
Fast Recovery Algorithm, Standard RFC 6582, Apr. 2012. [Online]. Available:
[Link]
[23] A. Sivaraman, K. Winstein, P. Thaker, and H. Balakrishnan, ‘‘An experi mental study of
the learnability of congestion control,’’ ACM SIGCOMM Comput. Commun. Rev., vol. 44,
no. 4, pp. 479–490, Aug. 2014, doi: 10.1145/2740070.2626324.
[24] W. Wei, H. Gu, and B. Li, ‘‘Congestion control: A renaissance with machine learning,’’
IEEE Netw., vol. 35, no. 4, pp. 262–269, Jul. 2021.
[27] F. Y. Yan et al., ‘‘Pantheon: The training ground for Internet congestion-control
research,’’ in Proc. USENIX Annu. Tech. Conf. (ATC), 2018, pp. 731–743. [Online].
Available: [Link] org/conference/atc18/presentation/yan-francis
[28] N. Jay, N. Rotman, B. Godfrey, M. Schapira, and A. Tamar, ‘‘A deep reinforcement
learning perspective on Internet congestion control,’’ in Proc. 36th Int. Conf. Mach. Learn., in
Proceedings of Machine Learning Research, vol. 97, K. Chaudhuri and R. Salakhutdinov,
Eds., 2019, pp. 3050–3059. [Online]. Available: https://
[Link]/v97/[Link]
18
[30] V. Sivakumar, O. Delalleau, T. Rocktäschel, A. H. Miller, H. Küttler, N. Nardelli, M.
Rabbat, J. Pineau, and S. Riedel, ‘‘MVFST-RL: An asynchronous
RLframeworkforcongestioncontrol with delayed actions,’’ 2019, arXiv:1910.04054.
[31] S. Abbasloo, C.-Y. Yen, and H. J. Chao, ‘‘Classic meets modern: Apragmatic learning-
based congestion control for the Internet,’’ in Proc. Annu. Conf. ACM Special Interest Group
Data Commun. Appl., Technol., Archit., Protocols Comput. Commun., New York, NY, USA,
Jul. 2020, pp. 632–647, doi: 10.1145/3387514.3405892.
[32] W. Li, F. Zhou, K. R. Chowdhury, and W. Meleis, ‘‘QTCP: Adaptive congestion control
with reinforcement learning,’’ IEEE Trans. Netw. Sci. Eng., vol. 6, no. 3, pp. 445–458, Jul.
2019.
[33] K. Xiao, S. Mao, and J. K. Tugnait, ‘‘TCP-drinc: Smart congestion control based on
deep reinforcement learning,’’ IEEE Access, vol. 7, pp. 11892–11904, 2019.
[34] W. Li, H. Zhang, S. Gao, C. Xue, X. Wang, and S. Lu, ‘‘SmartCC: Areinforcement
learning approach for multipath TCP congestion control in heterogeneous networks,’’ IEEE J.
Sel. Areas Commun., vol. 37, no. 11, pp. 2621–2633, Nov. 2019.
[35] A. Sacco, M. Flocco, F. Esposito, and G. Marchetto, ‘‘Owl: Congestion control with
partially invisible networks via reinforcement learning,’’ in Proc. IEEE Conf. Comput.
Commun., May 2021, pp. 1–10.
[36] X. Liao, H. Tian, C. Zeng, X. Wan, and K. Chen, ‘‘Astraea: Towards fair and efficient
learning-based congestion control,’’ in Proc. 19th Eur. Conf. Comput. Syst., vol. 4, New
York, NY, USA, Apr. 2024, pp. 99–114, doi: 10.1145/3627703.3650069.
[37] L. Roychoudhuri and E. S. Al-Shaer, ‘‘Real-time packet loss prediction based on end-to-
end delay variation,’’ IEEE Trans. Netw. Service Manage., vol. 2, no. 1, pp. 29–38, Nov.
2005.
19