Althea Development Update #62: Free tier design and the life of a packet
Since our last update bandwidth payments have been making their way down the stabilization pipeline we’re going to roll out Alpha 12 to our networks shortly after Christmas. At which point all routers will be paying for their internet access in Rinkeby Eth that we will airdrop into existing networks.
The Network DAO Aragon organization has also made significant progress it’s available on Rinkeby here. Feel free to play around with it.
Our next release is going to be the big one. Beta 1, which is by definition feature complete and ready to be used (cautiously) with real money in the real world.
Implementing the free tier
The last major feature we need to implement to reach Beta 1 is enforcement. What you actually do if people don’t pay.
Althea doesn’t ever actually cut anyone off, a free tier with limited bandwidth is provided gratis to help users get setup and generally use the internet. All the routers in the network agree not to bill each other for the free tier level of traffic.
The exact speed of the free tier is decided by the local network organization.
Having the free tier be enforced with a speed limit at all is a little anachronistic for Althea. Like I explained in our billing blog post connection speeds are hardly reflective of internet usage experience.
But it’s much easier to implement a speed limit than it is to make layers of prioritization based on bidding.
The life of a packet in an Althea network
I’ll expand on each part of this diagram in later blog posts. But to focus in on what we’re changing in Beta 1 lets talk traffic queuing.
In the Alpha 12 diagram the only queue operation we have is fq_codel, which is a traffic prioritization algorithm designed to try and reduce latency and make everyone’s connection ‘feel fast’.
If you have any interest in the plumbing of the internet I recommend this talk from around the time Codel was first being designed. It’s an excelent source of background, intuition, and tooling.
Bufferbloat: Dark Buffers in the Internet
As powerful as Codel-derived tools have become since that talk a careful reading of the documentation will make it obvious that a few suboptimal decisions where made in this first network design.
The codel operator on the router side doesn’t actually serve any useful purpose as it’s can’t identify connections because of the encryption.
Likewise the codel queue operation on the exit side isn’t ideally placed. While this one can at least see the traffic it’s supposed to be managing that management is performed over every connection coming in and out of the exit.
This can lead to globally optimal decisions but is less sensitive to a single routers problems.
Our new design for Beta 1 moves the codel operators so that they have a better view of traffic to manage and are generally closer to the ‘edge’ of the network.
With one placed in front of the router LAN and the other placed after mesh routing in order to prevent a single router from hogging traffic across the network.
I’ll be keeping a close eye on the new codel layout and hopefully find ineffective operations a little faster this time.
The other addition to our queue system is HTB (Hierarchical token bucket, no relation to crypto tokens), which is a pretty old fashioned bandwidth allocator. Using HTB we can reduce peers to the free tier level of bandwidth.
Sadly HTB isn’t really flexible enough to operate across interfaces. Meaning we can’t pool multiple interfaces worth of free tier bandwidth in such a way that also makes sense from a billing perspective.
Like I said before the use of a speed in the free tier system at all is not ideal, but it will do until we can implement a more elegant way of accounting for ‘unsold’ traffic.