If you’ve ever dealt with Unreal Pixel Streaming you know that improving the network performance can be a real pain. From optimizing the in-game environment to assigning the right cloud resources, it’s a game of fine-tuning till it feels right for your players.
We came across an issue in our play tests, where players couldn’t get past the point of the ‘WebRTC Connection Negotiated’ in their browser.
According to Epic’s own documentation, when a connection to a pixel streaming server is made from a browser
“each party needs to find out its own publicly visible IP address by querying a server that implements the STUN (Session Traversal Utilities for NAT) protocol. After the STUN server tells each endpoint its publicly visible IP address, the Signaling and Web Server can continue brokering their direct connection.”
Epic
The default configuration that we’ve been using is with Google’s STUN server.
In this setup some players could not connect occasionally, others could not enter our game at all.
The answer turned out to be in NAT (Network Address Translation).
Huawei.com
- Full-cone NAT
All requests from the same private IP address and port (for example, IP1:Port1) are mapped to the same public IP address and port (for example, IP:Port). In addition, any host on the Internet can communicate with the host on the intranet by sending packets to the mapped public IP address and port.This is a relatively loose NAT policy. As long as the mapping between the private IP address and port and the public IP address and port is established, any host on the Internet can access the host on the intranet through the NAT device.- Restricted-cone NAT
All requests from the same private IP address and port (for example, IP1:Port1) are mapped to the same public IP address and port (for example, IP:Port). A host on the Internet can send packets to the host on the intranet only if the host on the intranet has previously sent a packet to the host on the Internet.- Port-restricted cone NAT
Port-restricted cone NAT is similar to restricted-cone NAT, but the restriction includes port numbers. That is, a host on the Internet (for example, IP2:Port2) can send packets to a host on the intranet only if the host on the intranet has previously sent a packet to the host on the Internet.- Symmetric NAT
All requests sent from the same private IP address and port to a specific destination IP address and port are mapped to the same IP address and port. If a host sends a packet with the same source IP address and port number to a different destination, a different NAT mapping is used. In addition, only a host on the Internet that receives a packet from a host on the intranet can send a packet back.Unlike port-restricted cone NAT that maps all requests from the same private IP address and port to the same public IP address and port, regardless of their destinations, symmetric NAT maps requests with the same source IP address and port number but different destinations to different public IP addresses and ports.
To break it down for our issue: Cone allows external hosts to send to anybody under NAT. Symmetric allows external hosts to send to only somebody under NAT, from whom it already received a packet.
For the player that could get in occassionally (player A) it could be an Address Restricted or Port Restricted subtype of Cone NAT, after some tries the player succeeded with peer discovery
Our other player (player B) could not get on at all.
STUN fails to provide a valid address if the browser is under symmetric NAT.
“Server Reflexive”
Those addresses work depending on the topological conditions of the network. Therefore, STUN by itself cannot provide a complete solution for NAT traversal.
“Peer Reflexive”
TURN (Traversal Using Relays around NAT) is a protocol that assists in traversal of NATs or firewalls. It is most useful for clients on networks masqueraded by symmetric NAT devices.
It is a relay that wedges between the streamer and the client.
TURN gets around even symmetric NATs because both the client and peer can at least talk to the TURN server, which has allocated a relay IP address for communication.
There is no bandwidth difference from our infra point-of-view as clients would connect to the Pixel Streaming server directly anyway after a successful STUN discovery.
The bandwidth from Pixel Streaming to TURN happens on an internal network.
Downside is that processing power is used by the TURN server.
Epic references coturn, an open source TURN and STUN server. It can be configured to offer both at the same endpoint.
“The STUN and TURN protocols together, along with the ability to fall back from one server to another, make up the ICE (Interactive Connectivity Establishment) framework.”
Epic