Every time you load a website, your data takes a physical path across real cables, switches, and routers, owned by different networks that have agreed, in advance, on who will carry what. This post walks through that entire journey, then digs into the infrastructure that shapes it: Internet Exchange Points (IXPs), community IXPs, regional peering, and the difference between free and paid interconnection.
The Journey of a Web Request
Sending a letter to your next-door neighbor is fast, you just walk it over. Sending a letter overseas means it goes through your local post office, then a regional sorting center, then an international hub, then the same in reverse on the other end. Same letter, very different number of stops.
When you type a URL or click a link, your device first looks up the site’s address (DNS), then sends a request out through your home or office network to your Internet Service Provider (ISP). From there, what happens next depends entirely on where the website is actually hosted, and what relationship your ISP has with the network hosting it.
Visiting a local website: Suppose the site is hosted by a company in your own city or country, and that hosting company’s network happens to be a peer of your ISP, or even a paying downstream customer of your ISP. In that case, your ISP already has a direct route to it. The request goes straight from your ISP to the destination network, often through a local IXP, and the response comes straight back the same way.
Visiting an international website: Now suppose the site is hosted on a large cloud platform in another country. Your ISP almost certainly does not peer directly with that specific destination network, since peering only covers networks it has a direct arrangement with. Instead, your ISP hands the request up to its own upstream provider. If that upstream provider does not have a direct route either, it hands the request further up to a larger provider, and so on, until the request reaches a network that does have a path to the destination, whether through its own peering or its own paying customers. The response then flows back down through that same chain.
Every hop in that chain is a decision made ahead of time, through the peering, upstream, and downstream relationships covered in this post’s companion piece on ASN relationships. The number of hops and the physical distance covered both add delay, which is exactly why the infrastructure discussed below (IXPs, community IXPs, and regional peering) exists: to make the “local website” path as common as possible, and to shorten the “international website” path wherever it can be shortened.
Where IXPs Fit In
An IXP is like the landlord of a shared meeting hall. It provides the room, the tables, and the chairs so different businesses can sit down and talk to each other. The landlord doesn’t take part in any of those conversations or decide who agrees to what.
An IXP is not a network in the ASN sense. It has no address space of its own and does not carry traffic beyond its switch. It is simply a shared switch fabric, sitting in a data center, that many networks connect to so they can set up peering sessions with each other without each needing a separate cable and connection to every other network.
The peering itself, deciding which routes get shared with whom, still happens between the networks, either directly (bilateral) or through the IXP’s route server (multilateral). The IXP just provides the wires and the switch.
Community Peering (Multilateral Peering)
Instead of every shop owner on a street running a private phone line to every other shop, they all plug into one shared switchboard in the middle of the street. Anyone plugged into that switchboard can reach any other shop plugged in, without needing a separate line to each one.
Peering can happen as a one-to-one deal between two networks, but most peering today happens through an IXP’s shared switch. Instead of running a separate cable and BGP session to each peer individually, a network connects once to the IXP and can exchange routes with many other members through a shared route server. This is often called multilateral or community peering.
Settlement-Free Peering vs. Paid Transit
Settlement-free peering is like two friends splitting a lunch bill evenly, because they both eat about the same amount. If one friend starts eating noticeably more, it’s reasonable for the other to ask them to start paying their share, that’s paid peering. If someone wants a meal service that delivers absolutely anywhere, not just to that one friend’s kitchen, they need a subscription instead, that’s paid transit.
Peering is usually described as free, but the more precise term is settlement-free, meaning neither side settles a bill with the other. This works when the traffic flowing each direction between the two networks is roughly balanced, so both sides are getting roughly equal value out of the arrangement.
Paid transit (the upstream/downstream relationship) is the opposite: one side pays, and in exchange gets access to the entire internet, not just to one network’s own address space.
There is also a middle ground, sometimes called paid peering. This happens when two networks want a direct connection, but their traffic is too imbalanced, or one side is seen as gaining more value than the other, for a settlement-free deal to make sense. Instead of buying full transit, the smaller or more one-sided network pays a fee for a direct peering connection, limited to that one network’s address space, without the broader reach of full transit.
| Arrangement | Reach | Payment |
|---|---|---|
| Settlement-free peering | Only each side’s own address space | None |
| Paid peering | Only each side’s own address space | One side pays a fee for the direct link |
| Paid transit (upstream/downstream) | The entire internet | Downstream pays upstream |
How Commercial IXPs Make Money
Most IXPs earn revenue through a few main fees charged to the networks that connect:
- Port fees: A recurring charge for a port on the exchange switch, priced by speed. A 100 Gbps port costs more than a 1 Gbps port.
- Colocation fees: Charges for housing a network’s router or server equipment in the IXP’s data center.
- Cross-connect fees: The physical cable connecting a member’s equipment rack to the IXP’s switch.
- Value-added services: Some IXPs also charge for extras like route server access, DDoS mitigation, or private interconnects between specific members.
Large commercial IXP operators run this as a for-profit business, similar to renting out space and infrastructure.
What Is a Community IXPy?
A community IXP is typically structured as a non-profit member association or cooperative, run by and for the local networks that use it, rather than as a company seeking to earn a profit. A board elected from the membership usually sets policy and pricing.
Community IXPs still typically charge port fees, but priced at cost recovery rather than profit. Funding often also comes from:
- Membership dues from participating networks
- Grants from organizations like the Internet Society
- Support from local governments, universities, or national research and education networks
Why Community IXPs Are a Big Deal
It’s a bit like a small town finally getting its own train station instead of everyone having to drive to a station three towns over. Once the station exists, local businesses build up around it, and it becomes easier and cheaper for everyone to get around, not just the first people who asked for it.
A well-run community IXP tends to create benefits well beyond the networks that first join it:
- Keeps money local: Local-to-local traffic no longer has to be routed through paid international transit, so that money stays in the local economy instead of flowing out to foreign providers.
- Attracts content delivery networks: Once an IXP reaches a critical mass of local traffic, large content platforms and CDNs (think video streaming, search, and cloud providers) often deploy caching servers directly at the exchange. That makes popular content faster for everyone, and even cheaper, without those networks needing their own separate deal with each cache operator.
- Improves resilience: Local traffic keeps flowing even if an international submarine cable or long-haul link is cut, congested, or degraded.
- Grows the local tech ecosystem: Running and maintaining an IXP builds local networking expertise, and cheaper, faster local interconnection makes it more viable to host content and services locally rather than abroad.
- Encourages more local peering over time: As more networks join and see the benefit, the exchange tends to grow, further shortening paths for everyone already connected.
This is a large part of why organizations like the Internet Society have historically funded and supported community IXP projects, particularly in regions that are still building out their internet infrastructure.
What Latency Is, and Why It Matters
Latency is the time it takes for a piece of data to travel from one point to another and back, usually measured in milliseconds. It is not about how much data can move (that’s bandwidth), it’s about how quickly a single request gets a response.
Latency is largely a function of physical distance and the number of stops (hops) traffic has to make along the way. Every extra hop through an intermediate network adds processing time, and every extra mile of distance adds unavoidable delay, since data cannot travel faster than the speed of light through fiber.
It matters most for anything interactive: video calls, online gaming, voice calls, and even how fast a webpage feels like it loads.
Latency vs Bandwidth
Latency is how long it takes for one piece of data to make a round trip, measured in milliseconds. It is about delay, not size.
Bandwidth is how much data can move per second, measured in Mbps or Gbps. It is about capacity, not speed of a single request.
A simple way to see the difference: bandwidth is the width of a pipe, latency is how long it takes water to travel through it. A wider pipe (more bandwidth) lets more water flow at once, but it does not make the water at the far end arrive any sooner if the pipe is very long (high latency).
Practical example: streaming a large video file mostly needs high bandwidth, since a lot of data has to move. A video call or online game mostly needs low latency, since delay in a single exchange is what causes lag, even if plenty of bandwidth is available.
You can have high bandwidth with high latency (a satellite internet connection, for example), or low bandwidth with low latency (an old dial-up line to a very nearby server). They are independent measurements.
How Peering Helps Latency
When two networks peer directly (whether bilaterally or through an IXP), traffic between them takes the shortest possible path: straight from one network to the other. Without peering, that same traffic might have to travel up through one or more upstream providers, possibly to a completely different city or country, before coming back down to reach its destination.
Fewer hops and a shorter physical path both reduce round-trip time, which is why networks with a lot of traffic between them often seek out peering specifically to improve performance, not just to save on transit costs.
How Regional Peering Helps
Local, city-level peering solves the shortest-distance problem, but a lot of traffic is not quite local either, it is regional: two networks in neighboring countries, or across a wider geographic region, that still don’t need to route through a completely different continent to reach each other.
Without regional peering, traffic between two networks in, say, neighboring countries on the same continent might still be routed out to a distant international hub, simply because that is where their respective upstream providers happen to connect, and then routed back. Regional peering fabrics and regional peering forums, often supported by organizations like the Internet Society, exist to bring operators across a region together so they can build direct links with each other, without needing to route through a distant hub on another continent entirely.
The effect is similar to local peering, shorter paths, lower cost, better resilience, just applied at the scale of a region instead of a single city. It also helps smaller countries in particular, which may not have enough local traffic on their own to justify a large IXP, but can benefit significantly from connecting with neighboring countries’ exchanges.
This matters most in underserved regions, where a single country often does not generate enough local internet traffic on its own to attract large content platforms or CDNs to deploy servers there. By pooling traffic across several neighboring countries through regional peering, the combined volume becomes large enough to make it worthwhile for CDNs and cloud providers to locate infrastructure regionally instead of on a distant continent. That, in turn, gives every participating country lower latency and lower transit costs, benefits that would have been out of reach for any one of them alone.
How Community IXPs Help Latency
This effect is especially large in regions that don’t yet have strong local interconnection. Without a local IXP, two ISPs in the same city might have no direct path to each other at all, so traffic between them has to travel to a distant hub, often in another country or even another continent, and then back, purely because that’s where their upstream providers happen to connect.
A community IXP fixes this by giving local networks a place to peer with each other directly, in the same city or country. Traffic that used to take an international round trip can instead stay entirely local.
The practical result: lower latency, lower cost (since less traffic has to cross expensive international links), and more resilience, since local traffic keeps flowing even if an international link has problems.
Who Pays for Upstream at a Community IXP?
The IXP itself never pays for, or provides, upstream access. It is only a switch, with no reach to the wider internet on its own. Each member network that connects to the IXP still needs its own separate transit arrangement with a paid upstream provider, to reach everything it does not peer with directly.
The IXP’s port fee only covers access to the local peering fabric, exchanging routes with other connected members. It does not include, or substitute for, buying upstream. A network that joins a community IXP and buys no upstream at all would only be able to reach the other networks peering at that same exchange, nothing further.
Why CDNs Matter So Much to Community IXPs
Content Delivery Networks (CDNs) cache popular content, video, software updates, web pages, close to the people requesting it. Once a community IXP has enough combined local traffic, CDNs often find it worthwhile to place caching servers directly at that exchange.
This matters enormously for a community IXP’s value, because a large share of everyday internet traffic, streaming video especially, can then be served from that local cache instead of being pulled over paid international transit every single time. It is common for CDN traffic to end up as the majority of a mature community IXP’s total volume.
For the ISPs connected to the exchange, this directly cuts the amount of expensive upstream transit they need to buy, since so much of what people actually watch and download no longer has to travel overseas at all.