Transforming the intrinsic systems of funding and operating utilities and telecom is no small feat, nor is it a quick process. Since its development, the Althea protocol has changed how critical infrastructure is funded, owned, and operated. The protocol empowers operators of all sizes to build locally owned networks while returning choice for end users. Now with Althea L1, a purpose-built platform reliable enough to scale, the Althea protocol will finally be ready to reach its full potential. 

2025: A Year in Review

Over the course of the year, the Althea protocol and its ecosystem crossed several major milestones:

• Unlock of Althea L1, after a year of preparation Althea L1 is ready to enable new scale, reliability, and DeFi capabilities for Althea Protocol users.
• Deployment of the iFi DEX, an exchange purpose-built for the needs of utilities and telecom. Designed for efficiency, it features Liquidity Fusion, which combines ambient liquidity with knock-out and limit orders in a single pool.
• Community governance decisions, including votes to:
  • Manage the iFi DEX parameters
  • Launch an incentives program for the iFi DEX
  • Unlock transfers of the Althea token, now listed on Kraken
• Final prep of Althea L1 firmware for routers, enabling machine-to-machine payments directly between network devices using the Althea L1 blockchain.
• Expansion of Liquid Infrastructure, which uses the Althea L1 blockchain and the EVM (Ethereum Virtual Machine) compatibility to bring the power of DeFi financing to broadband networks. Read the Arizona case study.

Expanding the Stack: Cellular & Mobility

2025 also delivered meaningful progress in cellular infrastructure. Enhancements to the cellular stack now enable roaming between private and public mobile networks. Enabling unprecedented access to powerful cellular tools to build in the mobile space, use licensed frequencies, and connect a variety of devices. Contact the Althea Foundation if you are interested in building using roaming cellular on Althea L1 - more to come on this in 2026! 

Looking Ahead to 2026

In 2026, the work will continue to evolve Liquid Infrastructure for scale and to connect DeFi directly to telecom and utilities revenue.

Users of Althea L1 continue to innovate DeFi primitives with pooled Liquid Infrastructure Tokens (LITs) that represent diversified revenue streams. This could include applications such as lending or auction markets, or wrapped pooled tokens. Development of applications for Althea is permissionless, and the Althea L1 platform can be utilized for a wide variety of DeFi and RWA applications. Apps and tokens can be added to the althea.link front-end through token holder governance. 

Through real-world deployments and operator adoption, the Althea protocol has demonstrated a vital alternative to brittle legacy systems, one that preserves local ownership, user choice, and sovereignty. We build because this movement matters. The fundamentals are now in place, and the Althea protocol is ready to scale on its rightful home: the Althea L1 blockchain.

Participate in the discussion at forum.althea.net to add your voice to Althea’s evolution.

Funding, building, and operating infrastructure, especially in telecom, has long been brittle and broken, leading to unhappy users and operators struggling under unsustainable economics. Liquid Infrastructure was developed as a set of tools that enable programmatic revenue sharing among stakeholders, aligning incentives between users and operators to create more sustainable systems.

Already, Liquid Infrastructure is being deployed across the U.S. to build, operate, and expand networks in ways that preserve local sovereignty and management autonomy, resulting in more resilient operations and sustainable growth for all participants.

Evolution of Liquid Infrastructure

Liquid Infrastructure is defined by three key characteristics: on-chain, programmatic, and transparent. Revenue sharing occurs almost instantly as income flows into the LISC (Liquid Infrastructure Smart Contract) on Althea L1, directly from customers paying for services. This creates a seamless coordination mechanism among stakeholders, mitigating counterparty risk and enabling operational flexibility.

These on-chain fundamentals also support the evolution of LIT (Liquid Infrastructure Tokens) as usage scales. Future developments could include the creation of on-chain markets for buying, selling, and lending LIT, such as auction modules or decentralized lending pools, enhancing liquidity and capital efficiency.

As a native Althea L1 primitive, Liquid Infrastructure is open to development teams building applications and user interfaces. It can be adapted to a wide range of use cases, including data centers, large-scale infrastructure projects, metered billing in energy grids, fleet coordination, and vending network management.

Each LIT is specific to a given deployment and fully transferable, unlocking the potential for secondary markets and asset exchanges. Historically, telecom and fixed assets have been discounted due to illiquidity, and evolving Liquid Infrastructure into a framework that includes on-chain auctions and marketplaces could transform the sector and drive new growth.

DeFi Meets Real-World Assets (RWA)

The programmable nature of LIT on Althea L1 introduces new DeFi applications, such as lending, applied for the first time to telecom and infrastructure assets. Just as RWA assets like Treasury notes are used in on-chain lending protocols, LIT can bring telecom revenues into the DeFi ecosystem. This removes the manual friction of traditional infrastructure financing, expanding growth and improving capital efficiency.

Althea L1, purpose-built for telecom and utilities, provides the ideal platform for this convergence of DeFi and infrastructure finance. It integrates machine-to-machine settlement directly into its architecture and prioritizes utility-grade transactions, avoiding the congestion and high fees typical of general-purpose blockchains.

Additionally, the iFi DEX (Infrastructure Finance Decentralized Exchange) introduces seamless stablecoin swaps and native integration with the Althea ecosystem. With these purpose-built tools, we can envision a future where DeFi’s flexibility and liquidity solve systemic inefficiencies in infrastructure finance, building a stronger foundation for the emerging machine economy.

Next Steps

The upcoming Cardinal Update marks the next milestone in Althea L1’s evolution. This update introduces:

• Enhanced UX and governance capabilities for the iFi DEX
• Improved support for machine-to-machine payments
• Foundational upgrades for future DeFi applications built on Liquid Infrastructure

Auto Swapping and Stablecoin Fees: DeFi for Machines

This new feature provides a stablecoin fee only path for unwrapping tokens from the EVM. Tokens bridged into Althea L1 are automatically wrapped into the EVM for DeFi ease of use. For embedded devices with wallets using the Althea L1 that swap on the micro-payments layer, this upgrade allows those routers to pay USDC/stablecoins in fees directly. This increases reliability and reduces response time, making it seamless and practical for Machine-to-Machine devices to move funds into and out of the micro-transaction layer more frequently.

One of the biggest lessons we've learned designing M2M payment systems is that workflow complexity spirals fast. We designed universal permit tx support to allow M2M devices to make swaps without having to manage an ALTHEA balance themselves. Instead paying their fees in stablecoins to a relayer who pays fees in ALTHEA on their behalf.

But for a very common flow, like adding funds to an on chain router the complexity should be absolutely minimized for high reliability.

Governance for the iFi DEX

Initializes the NativeDex module to allow parameter and config changes to be sent to the iFi DEX via easy-to-use governance proposals. This means anyone can submit an easy to understand proposal to increase or decrease fees or change any other parameter in the iFi DEX. 

These updates support the routers and machines that use the micro-transaction platform and the usability of iFi DEX with Liquid Infrastructure.

Learn More about Althea L1 - https://blog.althea.net/althea-l1-a-focused-chain/

Liquid Infrastructure - https://medium.com/althea-mesh/introducing-liquid-infrastructure-a-new-defi-primitive-f4e2df14088b

iFi DEX - https://blog.althea.net/liquidity-fusion-in-the-ifi-dex-2/

Come Build With Us!

Are you building in the iFi space? We are actively looking for builders and projects to take part in an upcoming iFi whitepaper. Contact us at hello@althea.net.

• Discord: Join our community
• Learn More about how Althea works: website, docs
• Participate as a validator
• Launch an Althea network: Hawk Networks

Althea’s Liquid Infrastructure is redefining how broadband, cellular and infrastructure networks are funded by turning physical assets into blockchain-backed, revenue-generating digital instruments. Rather than relying on slow-moving grants or rigid bank loans, Liquid Infrastructure combines tokenized real-world assets (RWA), programmable finance, and instant revenue distribution to create a transparent, automated ecosystem for financing connectivity at scale.

Kamui Finance, a pioneer institutional investor in tokenized real world assets, led the investment backing the first large scale deployment in Liquid Infrastructure. The support of Kamui Finance represents growing institutional confidence in tokenized finance solutions.

“The Althea and Hawk Networks teams’ innovative, blockchain-based solution provides exactly the kinds of advantages we seek when investing in tokenized and on-chain finance. Transparent, predictable and automated distributions derived daily from infrastructure provide a higher level of investment security than typically available in traditional finance structures. We are proud to be able to support the deployment of Liquid Infrastructure RWA and support them in their future growth” said Hadi Kabalan, Director at Kamui Finance. 

Why a New Model Was Needed

Despite billions in federal and state broadband programs, many communities remain underserved. Traditional funding sources like grants, subsidies, and bank credit often come with delays, rigid terms, and bureaucratic hurdles that small and mid-sized ISPs struggle to navigate.

Broadband is one of the fastest-growing infrastructure sectors in the U.S., with market growth projected at ~10% annually, yet operators with strong customer loyalty and localized expertise are routinely locked out of capital markets. Althea’s Liquid Infrastructure closes this gap by making digital networks as investable and trackable as traditional financial assets—unlocking growth potential for both funders and operators.

How It Works

Liquid Infrastructure is a suite of blockchain-based tools that transform physical networks—fiber routes, towers, routers—into programmable financial assets:

1. NFT-Backed Assets: Each network or utility asset is represented by a non-fungible token (ERC721 NFT) on the Althea L1 blockchain, anchoring its identity and revenue stream on-chain.
2. Fractionalization via Smart Contracts: The Liquid Infrastructure Smart Contract (LISC) fractionalizes the NFT into fungible tokens, allowing multiple investors to share in top-line revenue.
3. Machine-to-Machine Billing: Althea’s payment system automatically collects user payments and distributes funds directly to token holders in near real-time.
4. Keystone Liquidity Platform: Hawk Networks’ interface for viewing performance metrics, claiming revenue, and managing tokens brings transparency and accessibility to both operators and investors.
5. Non-Custodial & Trustless: Smart contracts are non-upgradable and verifiable, giving investors direct control over their tokens and earnings without intermediaries.

Arizona Case Study

In Southern Arizona, a regional ISP with ~1,000 subscribers near a large market was able to scale operations and upgrade technology using Liquid Infrastructure. Hawk Networks leveraged blockchain-powered transparency and on-chain mechanics into tokenized RWA backed by institutional investors led by Kamui Finance to super-charge growth, add value to users and derive superior returns on investment. 

“What’s exciting about this model is that we can now align investment with impact. Local ISPs keep their identity and autonomy, while investors gain confidence from blockchain-level transparency. It’s a win for communities and capital,” said Deborah Simpier, CEO of Hawk Networks.

This approach has already accelerated the rollout of advanced connectivity solutions like KeyLTE/5G and Althea Mobile, proving Liquid Infrastructure’s value as a financing model for high growth areas.

Why It Matters

• For Investors: Liquid Infrastructure offers access to a $3T+ infrastructure market with unparalleled visibility and automated risk management.
• For Operators: Smaller ISPs finally have access to scalable, flexible funding that doesn’t compromise their independence.
• For Communities: This model prioritizes local agency and governance, while enabling faster deployments, with happier, more long-term customers.

Liquid Infrastructure shows that infrastructure can be financed as dynamically as it’s used, transforming broadband, utilities, and smart-city services into programmable financial ecosystems.

The Althea ecosystem has grown from a bold vision of decentralized infrastructure into a thriving global community of builders, validators, network operators, and believers. Today, we're proud to introduce The Althean Archetypes - a limited collection of 18 unique NFTs that celebrate the diverse roles and contributions that make our community extraordinary.

Ready to claim yours? Apply here 

Celebrating Every Kind of Althean

From the developers coding our iFi DEX to the technicians climbing towers in remote locations, from the validators securing our network to the voices spreading our message across social media - every Althean plays a vital role in building the future of Infrastructure Finance. These NFTs aren't just digital collectibles; they're recognition tokens that honor the real work our community does every day.

Each archetype represents a different way our community members contribute to Althea's mission of democratizing infrastructure and connectivity:

The Builder

"The future isn't just imagined, it's compiled."

The Builder forges the tools that power the machine economy. From crafting smart contracts for Liquid Infrastructure to optimizing our blockchain architecture, they transform ideas into code and possibilities into realities.

The Guardian

"Silent sentinels making decentralization possible."

The Guardian secures our digital cosmos as validators on Althea L1. Through constant vigilance and technical expertise, they ensure every micropayment, every governance vote, and every Liquid Infrastructure token mint happens safely and reliably.

The Weaver

"True decentralization happens one community, one installation, one connection at a time."

The Weaver transforms vision into connectivity. Whether they're planning network deployments, negotiating community partnerships, or physically installing infrastructure, they bridge the gap between digital innovation and real-world implementation.

The Explorer

"At the bleeding edge of innovation, venturing where others fear to tread."

The Explorer pushes the boundaries of what's possible with Infrastructure Finance. First to test new features, break things that need breaking, and experiment with ambitious deployments, they help shape our future through curiosity and courage.

The Voice

"The bridge between builders and believers."

The Voice makes complex technology accessible to all. Through social media, blog posts, community calls, and countless conversations, they translate technical innovation into human stories that inspire others to join our movement.

The Believer

"Seeing the impossible and making it inevitable."

The Believer envisions a world where infrastructure is truly decentralized - where smart cities run themselves, where communities own their connectivity, and where machine-to-machine payments enable new economic models. Their unwavering faith in our mission keeps the entire ecosystem moving forward.

Recognition, Not Sale

These NFTs are not for sale. Instead, we'll be distributing them on a case-by-case basis to recognize outstanding contributions to the Althea ecosystem. Whether you've been validating since our early days, building crucial infrastructure, creating educational content, or championing our vision in your community - your contributions matter, and these NFTs are our way of saying thank you.

Community members can receive multiple NFTs from the collection if their contributions span different archetypes. A developer who also runs a validator node might receive both The Builder and The Guardian. A network operator who creates educational content could earn both The Weaver and The Voice. The goal is to authentically recognize the diverse ways our community members contribute to Althea's success.

If you think you're eligible for one (or more), apply here.

Which Archetype Are You?

As we prepare to launch this collection, we're curious: which archetype resonates most with your contributions to Althea? Are you The Builder crafting the next generation of iFi applications? The Guardian securing our network? The Weaver connecting communities? The Explorer testing our latest innovations? The Voice spreading our message? Or The Believer holding the vision that guides us all?

These archetypes aren't just NFT categories - they're a reflection of the incredible diversity and dedication that makes the Althea community unique. Each represents a different path toward the same goal: building a more open, accessible, and decentralized future for connectivity and infrastructure.

Ready to join The Althean Archetypes?

Think you embody one of these archetypes? We want to recognize your contributions to the Althea ecosystem. Claim your NFT here

Follow our progress:

• Join our Discord: discord.gg/hHx7HxcycF
• Follow us on X: @AltheaNetwork
• Visit our forum: forum.althea.net

The iFi DEX

The iFi DEX is the foundation of DeFi on Althea L1, allowing users and machines to onboard and interact with Althea L1 and other protocols, such as Liquid Infrastructure, on the chain smoothly and automatically.

The DEX itself has been deployed and adopted by governance. The creation of new pools, and swapping on existing ones is permissionless with settings such as fees directly controlled by governance vote.

Currently v0.5 of Althea.link is deployed, with governance and staking functionality designed to operate while the chain was not yet unlocked. Version 1.0 of Althea.link, which includes bridging, LP, and swap pages has not yet been deployed for mainnet. You can check out v1.0 of Althea.link on the testnet at test.althea.link

Right now we're working on v1.5 of Althea.link, which dramatically simplifies the internal data handling and resolves some of the data dependency loops that can make v1.0 less responsive than we would like. Version 1.5 is expected to be the next version deployed live to Althea.link.

We're excited to take community feedback on what features we should be looking at in v2.0. Here are some that have already been proposed.

* More in depth trading view
* Chain activity and fees page

Althea Firmware

We've been working on a lot of new features for the Althea Firmware (for embedded devices like routers in telecom networks) in preparation for Althea L1. The most substantial feature is appliance exits which allows new networks to onboard to Althea L1 entirely out of the box, instead of requiring significant technical knowledge to setup an exit.

Now that the chain is unlocked and various constants like the stablecoin addresses and DEX addresses are known we can move forward with deploying the exit root of trust on Althea L1 and preparing firmware with out of the box configuration for Althea L1.

We hope to have images ready, tested, and distributed by the end of the month.

Althea L1 Relayer

We've talked about universal permit tx support before - this is an ecosystem of tools on Althea L1 that allows anyone to interact with the chain without having to already hold ALTHEA. This is essential for a chain with many machine users and real world applications, as acquiring some native token dust for fees can often be complicated so Althea L1 has an ecosystem wide solution.

That solution is the Althea L1 relayer. This program, which anyone can run to permissionlessly, relays gassless transactions, paying some ALTHEA in exchange for another token amount of slightly greater value.

These initial relayer tool is very straightforward and lacks fun features, like auto swap for the reward token back to ALTEHA, but is functionally complete. Looking forward to more polish and extra features in the next version.

Next up

If you're interested in more updates, have questions, or just want to interact with the Althea L1 community drop by the forum and discord!

Telecom systems—fiber, radios, routers—have long been treated as utilities. They move data, but they’ve had no agency, no embedded logic, and no role in finance. That’s no longer sustainable. If fintech is to reach its full potential—truly global, decentralized, and programmable—it needs infrastructure that can match that ambition.

Thanks to technology like Althea, today's networks are empowered to operate like economic agents. They can earn revenue, repay capital, split payments among stakeholders, and dynamically price services, all without human oversight. What was once passive infrastructure now plays an active role in creating and exchanging value.

We refer to this as resilient and sovereign infrastructure: networks that can govern, finance, and sustain themselves. It is infrastructure with financial logic built into the foundation.

iFi: When Infrastructure Becomes Financially Aware

Web3 takes the traditional fintech principles to new levels of autonomy and cooperation across a global landscape. At Althea, we’ve taken that concept further by developing a framework for Infrastructure Finance, or iFi.

Here’s how it works in practice:

1. A telecom network, such as a rural cellular deployment, is funded using on-chain capital.
2. These funders receive Liquid Infrastructure Tokens (LITs), which entitle them to a share of that network’s future revenue.
3. As users pay for service, the network streams small payments directly to all stakeholders: lenders, spectrum providers, equipment partners, and operators.
4. Once the target repayment is met, control of the asset shifts automatically—no manual billing, no reconciliation delays.

Each LIT is backed by actual usage and tied to the economic activity of a working network. These tokens reflect real performance, not speculation.

The Network Keeps the Books

Legacy infrastructure relies on layers of billing departments, contracts, and delayed settlements. Funds often move more slowly than the services they are intended to support.

Althea’s model removes those frictions. Revenue splits, debt repayment, and capital recycling are handled directly by the network itself. A router can send payments upstream as soon as a user connects. A radio can track its own payback schedule and fulfill it automatically. A network can continuously adjust how earnings are distributed based on performance, demand, or updated terms, without requiring human input.

With financial intelligence woven into the infrastructure layer, Althea enables systems to adapt, scale, and support new services as conditions evolve.

Building the Right Rails: The iFi DEX

To support this model of autonomous finance, we developed the iFi DEX, a decentralized exchange designed explicitly for infrastructure tokens, bandwidth markets, and machine-to-machine payments.

Off-the-shelf DEXs weren’t built for these kinds of users. Routers, sensors, and radios aren’t speculators—they need simple, predictable tools to manage cash flow and liquidity.

The iFi DEX is designed for that environment. It enables:

• Swaps of stablecoins and other revenue
• Real-time revenue distribution to token holders
• Programmatic settlement for services purchasing bandwidth

It also supports gasless transactions and stable, yield-bearing assets—critical features enabling machines, not just human traders, to interact with financial markets.

What This Unlocks

6G is often noted as the evolution that brings intelligence to the edge of connectivity. It enables dynamic and programmatic features on the radio that directly connect to the device. Instead of performing complex logic or operations at the data center or cloud, operations can be performed in real time, improving latency and maintaining the security and sovereignty of the data. 

Combined with Althea’s embedded financial design, we can envision key performance unlocks, such as dynamic spectrum sharing, supply chain tracking, or private cellular networks owned by multiple stakeholders.  

By embedding finance directly into the network’s operation, we make it possible to deploy high-performance infrastructure with less overhead, faster coordination, and a clear path to return on investment.

We’re already seeing this approach in action across apartment buildings, tribal lands, remote agricultural corridors, and dense urban environments. These aren’t pilots. These are active, growing networks built on principles of local ownership, open finance, and technological resilience.

Real Yield, Real Assets, Real Sovereignty

Infrastructure is no longer just a cost center or public utility. It’s becoming an asset class with programmable cash flow, clear performance metrics, and built-in governance.

This model changes who can deploy networks, how capital is raised and repaid, and the role connectivity plays in global finance.

The financial systems of tomorrow won’t just depend on infrastructure. They’ll be built directly inside it.

What Sets iFi DEX Apart

Althea L1 is focused on Infrastructure Finance (iFi), a new category of blockchain applications that combines concepts from DeFi (Decentralized Finance), Physical Infrastructure, and RWA (on-chain Real World Assets).  

Like most innovations, iFi builds on what came before it. One key foundational piece is a decentralized exchange (DEX) — essential for Althea L1 users to buy gas for EVM interactions, swap stablecoins for iFi transactions, and participate in DeFi built around iFi.

iFi DEX combines three liquidity types in a single pool through its Liquidity Fusion model: Each pool can be composed of ambient, concentrated and knockout liquidity—the latter also allows for limit order and for limit orders to function as liquidity. When all three work together, swaps are executed using the optimal mix of the three types of available liquidity and are executed in a single transaction. Liquidity Fusion enables greater efficiency, faster execution, lower gas costs while minimizing slippage. You can read more about how liquidity fusion pools work here.

Testnet Opportunities

iFi DEX will be launching soon, and we’re inviting you to participate in the upcoming testnet, scheduled to begin April 15th, 2025. As a testnet participant, you would test out the core features and functionalities of the iFi DEX. This includes…

• Test the Liquidity Fusion model firsthand
• Provide liquidity across all three types
• Execute swaps and measure system efficiency
• Shape the final product through direct feedback

Join as an iFi Explorer

Participation offers clear benefits:

• Early access to next-generation DEX technology
• Direct influence on infrastructure finance development
• Identification of improvements before mainnet
• Practical experience with production tools

How to Participate

Register using this form. 

Please also join the Althea L1 Discord server, where most of the Testnet coordination will be happening: invite link.

All participants will receive testnet connection instructions, test tokens, and access to documentation and support.

The Road Ahead

User feedback from the testnet will shape the final iFi DEX before its production launch. This completes a crucial piece of Althea L1's vision: bridging real-world assets with decentralized finance through a comprehensive infrastructure finance ecosystem.

Become an iFi Explorer and test the future of infrastructure finance on-chain with us.

Despite this untapped potential, the industry as a whole has thus far made surprisingly little progress in developing real-world M2M payment applications. While DeFi has made great strides in innovation, adoption and functionality, its focus remains centered on individual actions, not machines or automation. These priorities, while essential for human-driven finance, do not translate well to autonomous devices that require lightweight, continuous, and self-executing payment mechanisms.

Althea L1 is a purpose-built, scalable M2M payment solution, designed specifically to service applications like the Althea Protocol, which require secure, real-time, automated financial transactions between devices.

In this post we will talk about the software ecosystem that runs on Althea L1 and a little bit about the design of M2M applications themselves.

Billing vs Blockchain consensus

Over the last seven years spent developing and using the Althea routing and billing protocol, we’ve learned a lot about developing M2M applications. One of the biggest lessons is the tradeoff between on-chain and off-chain billing logic. 

Regardless of how payments are ultimately settled on Althea L1, development has focused on the balance of programmable blockchain logic that the M2M application uses and the way that it is embedded into the billing process. In general the more on-device logic is used, the more difficult application development will be. Blockchain consensus is extremely powerful, which is what makes consensus worth the tradeoffs around speed and cost of execution.The Althea protocol’s multi hop routing and billing system for internet payments is pretty far over to the left on this chart. It’s not until you climb up the stack into things like the exit layer or liquid infrastructure that the Althea protocol uses significant amounts of on-chain logic.

While this approach adds design complexity, it yields enormous scalability advantages. Althea protocol could scale to handle the traffic and payments of the entire global internet, something infeasible for an architecture relying solely on on-chain logic.

No M2M application on Althea L1 will be entirely on one side or the other of this spectrum, but it’s important to keep the spectrum in mind when working on M2M application design. This insight is what led to Althea L1’s hybrid design, where microtransactions for on-device logic are prioritized, with programmable EVM logic running at a slower pace. 

This tradeoff makes Althea L1 significantly more appropriate for applications generating revenue rather than applications focusing on blockchain incentives for real-world goods.

Incentive mechanisms are extremely chain-logic heavy by nature and inherently constrain the capacity of their host chain for M2M applications that focus on IRL revenue, which generally cannot be proven on chain. 

As an example, imagine a vending machine. On chain proof of sold candy bars is essentially impossible. But using on device logic the vending machine can determine it has been paid and release the item to the customer.

More powerful applications require better tools

Developing M2M applications that rely heavily on on-chain logic plays to the strengths of general-purpose Layer 1 blockchains—primarily because blockchain developer tooling is far more robust than embedded device tooling. However, this convenience comes at a cost as discussed in the previous section: scalability and efficiency.This challenge became especially clear during the Althea protocol’s development. Existing tooling simply did not provide the level of control needed to implement Althea’s device-centric design. Without off-the-shelf solutions to turn to, we had to build our own libraries for interacting with both EVM and CosmosSDK-based chains.

Clarity

Clarity is an ultra-lightweight, ultra-portable transaction generation library for the EVM that is fully compliant with the Ethereum consensus client test suite.

What makes Clarity unique is that it is fully cross-platform, meaning it is free of any CPU architecture-specific instructions across its entire implementation. We originally required this to support routers using MIPS CPUs, and while the rise of ARM powered hardware means this is no longer mandatory, It continues to be useful when porting to unique architectures and maintaining lightweight operation.Clarity is also extremely light weight, with only 36 dependencies it is ideal for embedded environments down to as little as a few hundred kilobytes of ram. 

Web30

Web30 is the request library companion for Clarity. Handling the network functions and jsonrpc interaction. The key design aspect of Web30 is a focus on robust response to a variety of network outages and conditions.Web30 exposes all network calls to the user in such a way that any failure can be handled gracefully with a complete understanding by the caller of how and when transactions are signed, submitted, and executed. This is a problem with other request libraries that include a very high level of abstraction—making it difficult, or even impossible, to handle a network failure at the wrong time, or an unexpected disconnection.

Deep Space

Deep Space was the first CosmosSDK-compatible transaction generation library to use Rust and also the first to make exclusive use of the GRPC interface for communicating with full nodes.

Unlike Clarity and Web30, Deep Space is a single library, because transaction encoding in CosmosSDK requires protobuf definitions that already pulled in network libraries there was no point in separating them.

Deep Space is the most performant library for interacting with CosmosSDK chains, capable of indexing 10 million blocks in roughly 100 minutes, utilizing even mediocre hardware.

Ecosystem infrastructure

In addition to the foundational libraries, Althea L1 includes several pieces of infrastructure designed for easy integration by M2M applications.

The iFi Dex

Every design choice for the iFi DEX was made in favor of optimization for M2M devices, payments and swaps. The topic of the high-speed backend is covered in detail in our introductory blog post.Higher speed and lower resource usage are nice to have for a DeFi platform, but they are essential for M2M applications that should successfully build swaps themselves rather than relying on less reliable centralized infrastructure.

For Althea L1 applications, the swap complexity is pushed to the DEX side. By taking advantage of the multiple liquidity types aggregated using Liquidity Fusion, devices can perform a single swap that is both routed across many different types of liquidity and potentially multiple pools without using a separate dex aggregator —thereby eliminating significant complexity from the device side of the equation.

Permit TX Aggregator

All DeFi infrastructure on Althea L1 supports Permit TX, which allows ALTHEA gas to be paid on your behalf by a relayer. This means that M2M application developers can spend less time writing gas handling code and more time working on their application.

Liquid Infrastructure

Liquid Infrastructure is the vanguard feature of Althea L1’s iFi DEX, enabling seamless integration between on-device revenue streams and on-chain DeFi applications. Even more importantly, it achieves this without hindering the overall performance of the chain.

Conclusion

Althea L1’s ecosystem is purpose-built, from top to bottom, to support M2M applications, with a unique focus on revenue-generating devices and bringing off-chain businesses on-chain. The enticement to Web3 in this case is not through incentives—but by delivering a superior customer experience, superior performance, lower cost of operation, and deep integrating with decentralized finance (DeFi)—powering a new era of iFi (infrastructure finance), enabling fully on-chain businesses to flourish.

Three Types of Liquidity

1. Ambient Liquidity (Uniswap V2 Style)

What is it? This is the "always available" liquidity that spreads across the entire spectrum of possible prices.

Advantages: Trades can happen regardless of the price the pool is at 

Disadvantages: Liquidity providers are more exposed to impermanent loss. Traders can experience inefficient swaps unless pools are large.

2. Concentrated Liquidity (Uniswap V3 Style)

What is it? Rather than spreading liquidity across the entire spectrum of potential exchange rates, liquidity providers select a specific price range where they want their liquidity to be active. Providers can also overlap multiple liquidity positions, allowing for a more nuanced approach when deploying liquidity.

Advantages: This makes the provided liquidity more efficient. It’s only used for swaps in the desired price range. Traders experience very efficient swaps and liquidity providers have more control over their exposure.

Disadvantages: If the price moves outside of the range, the liquidity becomes inactive and swaps can not execute (unless/until the price returns within the specified range). Concentrated Liquidity is also more complicated to manage for liquidity providers.

3. Knockout Liquidity (Limit Orders as Liquidity, Ambient / Ambient derivatives)

What is it? Knockout liquidity is a concentrated liquidity position with a very narrow price range that starts as entirely one token, and is removed from the pool when it becomes entirely made up of the other token. What this means is that it acts as both a liquidity position and as a limit order. For example, if someone wants to swap a large amount of a token at a set price, they can create a knockout liquidity position. This enables buyers to execute potentially enormous trades without slippage as long as there is sufficient knockout liquidity to cover the trade. Knockout liquidity will continue to be used as pool liquidity as long as the price stays within the defined range (think fractions of a cent price change). Once the price of the pool moves through the full range of the knockout position, the liquidity is automatically removed.. This allows the creator of the position to come by and withdraw their swapped tokens, in addition to any fees they may have earned for providing liquidity while the knockout liquidity position was active. If the price fails to move through the full range of the knockout position, it is also possible to withdraw early and get a mix of tokens and fees, much like how concentrated burns work.

Advantages: Knockout liquidity allows buyers and sellers to make large trades hyper-efficiently, and also provides improved swap quality to regular traders at the same time. 

Disadvantages: Knockout liquidity is not an instant swap, it requires buyers or sellers willing to move the price from one side of the range limit to the other.

Liquidity Fusion: Ambient, Concentrated and Knockout Liquidity work together

Every pool in the iFi DEX uses liquidity fusion, applying the optimal mix of the three liquidity types to execute every swap. 

• If the trade is within range of a knockout liquidity position, it will be used first reducing the price impact to effectively zero.
  
  
• If there is no knockout liquidity available iFi DEX will utilize concentrated liquidity provided that the price is within the range of the available concentrated liquidity.
  
  
• If the price moves outside the concentrated liquidity range, iFi DEX can still execute the trade using the ambient liquidity in the pool, even if it’s less efficient.

Liquidity Fusion provides an experience for swappers that is equal to or better than even the best DEX aggregators. DEX aggregators direct your swap across multiple pools and multiple DEXs in order to generate the best price. Liquidity Fusion can do that all in a single pool, faster, requiring less gas, and also provides knockout liquidity.

Limit order functionality in other DEXs (e.g. Uniswap) is limited to signing a normal swap that will execute at a later time. Knockout liquidity allows the user to trade time for efficiency that DEXs ordinarily cannot provide.

Ambient liquidity rewards

The iFi DEX is an Ambient DEX fork, with a new and completely unique feature only in the iFi DEX fork is integrated liquidity rewards. Ambient or Uniswap v2 style liquidity has two major advantages. It’s simple for liquidity providers to manage, and it’s fungible which makes liquidity rewards much easier to provide. With a concentrated liquidity position, every position is unique and is making a unique amount of swap fees based on the exact price range that was selected. Furthermore optimal utilization of the liquidity requires the price range of the liquidity providers position to be adjusted frequently. This can be a very active operation requiring weekly or even daily attention.

Ambient style liquidity on the other hand is a set it and forget it operation. Everyone providing ambient liquidity to a pool will have the same outcome, and receive the same amount of fees per unit of liquidity.  Some protocols and communities continue to use ambient style liquidity exclusively despite its reduced efficiency because participation is so much easier.

With Liquidity Fusion in each iFi DEX pool there would normally not be many good reasons to use ambient liquidity. The more efficient concentrated liquidity positions would earn the vast majority of the fees and discourage less specialized and infrequently active liquidity providers from being involved at all. This is a problem, because ambient liquidity can act as a strategic reserve for trading when a large price swing reduces active concentrated and knockout liquidity.Using the iFi DEX incentives program, governance or any other sponsor can provide rewards to be distributed to all ambient liquidity participants in a pool, encouraging liquidity providers to participate and allowing traders to take full advantage of all types of liquidity. 

A focused DEX

Althea L1 is a platform focused on providing an efficient, effective, and easy infrastructure platform. To fit into that mold the iFi DEX is designed as a one stop shop that replaces an entire ecosystem of liquidity pools and DEX aggregation. With the iFi DEX applications on Althea L1 can focus on building effective and reliable infrastructure for their customers. 

• Althea L1: A layer 1 protocol designed for machine-to-machine payments
• Liquid Infrastructure: A native feature of Althea L1 that bridges the gap between RWA and DeFi
• Improved appliance exit architecture

Though the core feature set has grown and new functionalities added, the foundational design of the Althea Protocol has remained remarkably consistent since the original whitepaper seven years ago and the first operational prototype five years ago. 

This stability is a testament to the strength and foresight of our initial design.

A recent upgrade to appliance exits marks the most notable refinement in how Althea Networks operate, aligning even more closely with real-world operator needs.

Right now Althea networks are structured like this: 

Gateways provide an internet bridge that pulls nodes not physically connected to the network into the local network, where clients can talk to them. The Althea Whitepaper 2.0 provides further details on gateways and the network structure.

Bootstrapping Trust

Bandwidth flows like water in an Althea network. It automatically traverses the path of lowest cost and highest quality, shifting constantly to find the sweet spot for the user, balancing cost, throughput and latency.. Your home internet traffic could take any route, even several different routes, through the network throughout the day.

Privacy is of course an essential consideration for any network, as no one should rely on a network that allows others to snoop on their internet activity. All traffic on an Althea network, from source to exit node, is encrypted using Wireguard and all Althea routers come pre-programmed with the public encryption key of an exit node to avoid man-in-the-middle attacks.

While this approach to routing and security is excellent from a technical perspective, it has an unintended effect on network operator behavior, who are more inclined to use exit nodes that are already programmed into the firmware.

Although it is fairly straightforward to add a new or additional exit node to a router—and takes less than a minute to do—in practice no operator to date has elected to do this at scale. Anything beyond the absolute minimum amount of configuration per user is a burden best avoided. For this reason, operators routinely request that their exit node details are added to the default build of the firmware.

This isn’t ideal, both because building a new version of the firmware takes time, but also because it simply doesn’t scale operationally. An ever longer list of exits to select just increases the chance of error during setup.

Far, Far Away

While deploying an exit server with simple automation is not particularly complex, in practice we find that network operators tend to put more focus on network management and less on server infrastructure, often relying on others to set up and maintain an exit at a strategically convenient location.This approach works well in well connected areas, but we have found that setting up an exit in more remote regions of Africa or South America can be a bit more challenging.

If a network relies on exits that are too far away or hosted in cloud data centers, network performance can be significantly impacted. Optimizing exit placement becomes crucial, and tolerating a sub-optimally configured network is simply not viable.

Until now a pre-configured exit was typically shipped, or a system administrator would remotely configure one for a given deployment, but these are added steps that hinder the speed and ease of network deployment.

Compare this with the steps involved to setup a typical Althea router:

1) Add firmware (using the browser-based UI)

2) Explore Internet

The challenge therefore is to enable simple and optimized deployment to any new network, regardless of geography.

Appliance Exits, Bootstrapped by Althea L1

This brings us to our solution: appliance exits. With appliance exits, we areredefining what it means to run an Althea network. This innovation introduces a new type of Althea firmware—exit firmware—which transforms any compatible router, not just server hardware, into a fully functional exit.

For operators, this means seamless integration of the exit and gateway roles, allowing them to directly route and exit customer traffic without additional infrastructure.

This simplifies setup and operation, reduces hardware costs, and presents more options for customers, whose routers are free to roam to any other exit over the internet.

Gone are the days of shipping specialized servers for each deployment. Now, any off-the-shelf $30 WiFi router can be flashed as an exit in minutes.

Exit Database Smart Contract

In the traditional server-based exit model, each exit required a backend database (PostgreSQL) and an authentication method to verify users, such as an SMS API key or an email account.

For most users, this authentication step was the only time they even noticed they were connecting to an exit. Often, this was handled during installation, making it an invisible part of the Althea experience.

Because Althea clients can seamlessly roam between exits using a shared customer database, operators naturally gravitated toward a single, unified database. This ensured users could always switch to the best available exit without re-registering or any other additional steps.

To simplify this process, we designed a solution eliminating the need for individual exit registration. Instead of signing up for just one exit, users now instantly register for all possible exits.

Appliance exits leverage an exit database smart contract on Althea L1, serving as a decentralized user database. Clients register directly to the smart contract on the Althea L1 EVM, which allows any exit to serve any user simply by referring to the smart contract data. The exits themselves also register to the same smart contract, enabling clients to retrieve a list of available exits and seamlessly connect to the best one available.

This approach removes nearly all manual configuration from exit operations. Exit Appliance images can now simply start up and go—no extra setup required.

Solving Bootstrapping

Even with the exit database smart contract, bootstrapping is not entirely solved. Sure, once a client is online, they can simply query the smart contract and get a list of exits to trust, but they need to query this list before they can get online.To resolve this Catch-22, we mirrored the concept of https certificate chains and created root trust servers for Althea exits. 

Instead of being built with the exit public key, Althea router firmware is now built with a number of root keys. The root trust servers simply grab the latest copy of the exit database smart contract data off of the Althea L1 EVM, sign it, and pass this data onto an exit. The exits hold these signed copies of the exit server list from the exit database and provide them to Althea routers as they come online. From there, the router can verify the signature, get online, and query updated data for themselves.

CGNAT and SNAT

In addition to the new appliance design, we’ve bundled significant feature improvements into this release. This includes support for CGNAT and SNAT IP assignment modes.For a quick introduction to why this matters, start with Why 32 Bits Wasn’t Enough, and the concept of NAT (Network Address Translation).Exits can now use multiple external IPv4 addresses in tandem and assign users their exclusive static IPv4 address out of the pool while doing it. Even better, this can all be configured from the router dashboard like any other feature.This is a huge jump for exit operators in terms of ease of use. Before the appliance exits, there was no GUI of any kind and a new exit would need to be added for every 400 or so Althea users that were connected to it.

With this new feature improvement, there is no longer any practical limitation to the number of users connected to a single exit in terms of software, as long as sufficient IPv4 space is available.

Native IPv6 has been supported since Beta 20 and will also be possible to configure from the UI.

Conclusion

Althea has always been about helping people build systems that work for them. We are excited to apply five years of operational experience to design refinement, optimization and simplification: making it faster and easier for our network operators to building new networks, deploy new infrastructure, in new and more challenging places with more customers than ever before!

Coming up next week is a deep dive on *Liquid Infrastructure* and exactly how the account abstraction on Althea L1 is designed.

A key challenge in this design is preventing both users and automated systems from being stranded without ALTHEA tokens to pay gas. No one wants a smart electric car charger that requires "dust" (tiny amounts of gas tokens) before it can function.

The Althea L1 Whitepaper covers why paying gas in ALTHEA is essential to operate a sustainable operating environment in a programmable blockchain.Given these requirements we set out to ensure that every component in the Althea L1 ecosystem supported delegated gas (permit) transactions. Where a user or machine can create and sign a transaction that pays a tip, using the tokens they have, to a relayer who pays the actual fee in ALTHEA when submitting the transaction.

This post will cover the various components and strategies we have implemented to enable users to pay transaction fees with stablecoins, or a token of their choice at every step of their on-chain journey.

Bridge In, Bridge Out

As discussed in the original Althea L1 design blog, the chain features two execution environments that coexist within a single blockchain. When a bridge transfer arrives on Althea L1—whether from Ethereum via Gravity Bridge or from any IBC-compatible chain—tokens from that transaction can be used in either the MicroTX environment or within the EVM.

The MicroTX module on Althea L1 directly accepts governance-approved stablecoins as fees. This allows automated machine-to-machine payments to begin immediately upon bridging, with no need for token swaps or additional actions.

All other transactions do require ALTHEA for fees. These are handled through either EIP712 permit transactions or Cosmos SDK transaction tips. Both of these are methods where a user can sign a tx paying fees in a token of their choice and have a relayer submit the actual transaction.

This is where EIP-712 is applicable. Both the iFi DEX and the native ERC-20 implementations for bridged tokens support delegated transaction fees, allowing users to pay for swaps using the assets they’ve bridged in—without needing to acquire ALTHEA first.

If a user only needs to perform a single swap and then exit the EVM, they can swap and return to the MicroTx layer without directly paying gas fees. Once out of the EVM, users can bridge out of the chain using a Cosmos SDK transaction tip funded IBC send.The iFi DEX is just the first DeFi application on Althea L1. Moving forward, we strongly encourage all iFi ecosystem applications to implement the EIP-712 standard and integrate with Althea L1’s relaying infrastructure.

Canonical Relaying Infrastructure

Without an active marketplace of transaction relayers, EIP-712 or Cosmos tip transactions wouldn’t be that practical.In the past the relaying infrastructure like this has been exclusively the domain of the app operator. With each app operator making their own infrastructure and backend to execute the transactions. For Althea L1 this is insufficient. 

Althea L1 users can submit their transaction to any one of several “indexing” endpoints. These endpoints do not perform the relaying themselves. Instead, they validate transactions and serve the set of submitted transactions for any relayer to request and relay.

By creating this separation between the relayer and the indexer, and providing them as infrastructure rather than a bespoke part of an application, relayers can easily enter and exit a market that represents the combined relaying needs of all applications on Althea L1 with no significant barrier to entry beyond setting up the reference relayer implementation.

Indexers, on the other hand, offer a low-cost public service by simplifying the user experience. Instead of requiring users to hunt for active relayers, they can submit their transaction to one or two indexers and be assured that it will reach every available relayer.

On-Chain Indexer List

Althea L1 governance can provide a canonical list of active indexers through a simple smart contract. This allows DeFi applications on Althea L1 to fetch the latest indexer list dynamically, rather than hardcoding a list of indexers that may change over time.

This ensures long-term reliability and makes it easier for new relayers to integrate seamlessly into the network.

Conclusion

Building sustainable blockchain infrastructure requires long-term vision. Instead of rushing to launch quick fixes like a token faucet, Althea L1 is creating an ecosystem where permit transactions are seamlessly integrated at every level and the flow of these transactions is open to a relayer community.

A true machine economy demands automation, reliability, and seamless execution. Each point of friction that requires human intervention diminishes the value of decentralized innovation.

By designing long-term solutions for fee payments, relaying, and governance, Althea L1 ensures that developers can build high-reliability applications—without being blocked by bootstrapping limitations.

What Do You Mean by “Open Rewards”?

Many DEXes incorporate rewards mechanisms to incentivize liquidity providers and bootstrap liquidity. What makes the iFi DEX rewards system unique is its open interface for third-party reward contracts.

Instead of defining a series of incentive parameters and building them into the main contract, the iFi DEX tracks metrics, such as liquidity provided and fees collected, and exposes them to any third-party contract that wishes to use them.The typical drawback of read-only interfaces is that rewards contracts must be triggered at precise moments in order to capture reward data. However, we have developed innovative ways to mitigate this limitation, while maintaining the flexibility of an external rewards system.

Delegated Liquidity Position Ownership

Ambient DEX supports using various account abstractions and external routers. This makes it possible to layer this functionality with the iFi rewards system—delivering an experience that is as simple to operate as a native rewards system, yet infinitely more flexible.From a user’s perspective, they simply create their liquidity position through the rewards contract, and call their rewards claiming function there. This flow allows the rewards contract to trigger data storage and snapshotting as needed, throughout the entire lifetime of the liquidity position.

Dealing With In-Range and Out-of-Range Positions

Due to Ambient’s highly efficient architecture, it is impractical to track which concentrated liquidity positions are in range at all times. Instead, we use concentrated liquidity fee collection—an already optimized process—as a proxy to determine if a position is in range.

Though it adds significant complexity to the rewards contract implementation, by exposing both the amount of concentrated liquidity and fees collected (on a per account basis), API callers can determine if a position was in-range and reward them appropriately.

This approach keeps the gas cost of these incentive extensions low—an approximate 9% increase in swap gas and 16% increase on LP position operations.

Rewards Contracts

As part of these changes to the iFi DEX we have two simple rewards contract implementations, one that focuses on incentivizing ambient liquidity in a pool and one that focuses on concentrated liquidity. Neither of these take into account fee collection. They provide simple incentives for liquidity which has been locked for a duration.

The implementation of both is straightforward. The contract owner creates and funds a rewards program which enables liquidity providers to register all their concentrated or ambient liquidity at once, then rewards accumulate for each registered liquidity provider until they are claimed. Liquidity providers must claim their rewards before changing their  positions, so that the rewards contract can be sure that no updates have been made to the liquidity provided and therefore can calculate rewards on the most recent liquidity amount.

In short, the flow is: Create LP -> register for rewards -> wait some time -> claim rewards -> modify LP if needed -> repeatThe concentrated liquidity rewards contract is only appropriate for stableswap pools, because it does not take into account fees or in-range positions. The ambient rewards can be applied to any type of pool.

The best thing about this split approach to rewards contracts is that rewards system upgrades are performed independently, without the need to upgrade to the main DEX contract. In fact, they do not require an upgrade to any contracts. The old rewards contracts are simply phased out and new ones are deployed. This design not only allows for faster iteration and greater flexibility, but also the power to run multiple distinct rewards programs, in parallel, on the same pool, to provide incentives for different types of behavior.

These reference contracts will be integrated with Althea L1 governance as part of the first-generation iFi DEX rewards program (after the proposal to operate a rewards program passes).

You can find these simple contracts here. 

Conclusion

We’re nearing the final installment of our iFi DEX blog series. Next week, we’ll dive into our last topic: EIP-712 transaction support!

Beyond configurability, the iFi DEX is also upgradeable, allowing for functionality expansions and bug fixes. Typically, such admin functions are managed by multisigs controlled by a handful of voters.

For the iFi DEX, we’ve taken a different approach. We’ve built a native governance bridge that makes all admin functions fully accessible through on-chain governance.

How Does Althea L1 Governance Work?

Althea L1 uses the Cosmos-SDK governance framework, which integrates governance with delegated proof-of-stake (DPOS) voting and staking. In other words, anyone who has staked/delegated tokens to a validator can vote on governance proposals, including proposals that are critical to DEX maintenance, management and growth.

This means that multitudes of stakers can now actively participate in iFi DEX governance decisions! This is a great improvement, but also one that presents a new problem: Enabling stakeholders to truly understand the proposals they are voting on.

Easily Readable Custom Proposal Types for iFi DEX Governance

The ability to see what bytecode a proposal passes to the iFi DEX does not make it inherently transparent, for not all users will be able to correctly interpret what they are looking at. To that end, we have integrated easily readable proposal types for the most important DEX functions.

High risk proposals, such as pausing DEX operation and adding or removing addresses that can change specific parameters, now have their own easily readable proposal types. Initially less critical functions, like simple pool fee changes, will still use standard proposal types, but can be upgraded to easily readable proposal functions in the future.

When a proposal passes on-chain natively, it calls the iFi DEX and applies the changes directly and automatically. There is no second step, or trusted party, required to carry out the voted upon changes.

What Are These Configuration Options, Anyway?

The iFi DEX is a fork of Ambient DEX, with several modifications. One of Ambient’s best properties is its stupendous configurability. For example, unlike Uniswap, it is possible to change many properties of pools after creation and overlay Uniswap v2 and v3 style liquidity positions in a single pool. This means that governance can fine-tune fee tiers or tick sizes in active pools, eliminating the painful need to create new pools and migrate liquidity between them to accommodate changing needs over the lifetime of a pool.

Conclusion

Decentralized governance is essential to making a DEX “canonical.” Since the iFi DEX is part of the chain itself, it needs to be held to the same governance structure and standards as Althea L1.

In addition to this alignment of standards and principles, distributed governance enables more advanced configurability options, which can be used with confidence by all stakers, eliminating reliance on a small, centralized group of decision makers.

Althea L1 is focused on Infrastructure Finance (iFi), a new category of blockchain applications that combines concepts from DeFi (Decentralized Finance), Physical Infrastructure, and RWA (on-chain Real World Assets).

Like most innovations, iFi builds on what came before it. One key foundational piece is a decentralized exchange (DEX)—essential for Althea L1 users to buy gas for EVM interactions, swap stablecoins for iFi transactions, and participate in DeFi built around iFi.

To power this, Ambient Finance was selected as a base due to its combination of concentrated liquidity, incredible feature density, and copyleft licensing. But we didn’t just stop there—we made major improvements to ensure the DEX fully aligns with Althea L1’s standards for reliability, decentralization, and performance.

This series of blog posts will introduce the iFi DEX. Althea L1’s canonical DEX. We’ve made significant changes to the Ambient base, focused on providing an experience that compliments iFi and meets the Althea ecosystem standard for reliability and decentralization.

Supercharging the DEX Backend

This first blog post in the series focuses on the Ambient graphcache-go backend and our work to create a drop-in replacement that syncs 1000x faster and uses 1000x less disk space. Every DEX interaction—whether querying a pool, performing a swap, or adding/removing liquidity—relies on the backend for data. Ambient Finance’s strength lies in its feature density and efficiency, but that also adds backend complexity. We tackled this challenge head-on, ensuring the iFi DEX backend is not only ultra-fast and scalable but also operationally seamless for users and developers alike.

Phenomenal Cosmic Power, Itty-Bitty Living Space

Ambient is a single-contract DEX, meaning all the pools and functionality exist as a single contract. Unlike decentralized exchanges such as Uniswap v2 where each pool is its own contract, Ambient pools exist as structured data in one giant contract. As deployed, the main Ambient contract is 23.824KB in size, just shy of the Solidity contract size limit of 24KB. This size does not include ‘sidecar’ contracts used to move code out of the main contract. Events are costly in terms of contract size. While the number of lines it may take to describe events is small, adding enough events to describe the full state of swaps and LP actions on the DEX would add roughly 1KB of storage to the main contract. Furthermore, this 1KB cannot be moved to sidecar contracts without resulting in a confusing experience where event consumers have to track events across multiple contracts.

Because of this, the canonical Ambient backend does not use events at all. Instead, it leverages an incredible level of EVM introspection provided by a Graph Protocol subgraph and a custom caching layer called graphcache-go. The subgraph doesn’t read the status of the DEX, it infers behavior directly from transaction stack traces and generates pseudo-events through a method much like forensic analysis of closed-source contracts. From this introspection, the graphcache-go backend can quickly and simply index the DEX history, simulate transactions to query live state, and generate up-to-date trading information. Additionally, graphcache-go stores historical information and serves most of its data as of a previous block upon request.This level of introspection comes at a steep operational cost, where graphcache-go and its subgraph: 

• Require an Archive Node for detailed transaction inspection
• Can only sync blocks serially in order, limiting parallel processing
• Must be fully resynced from scratch to compute new data

None of these are bad design decisions; they are the inevitable result of not having events. At first, this might not seem problematic. However, things go sideways when dealing with older instances of Ambient. By the time a DEX is just 2-3 years old, it can take more than a month to sync a new graphcache-go instance from scratch.This places enormous centralizing pressure on operating an Ambient frontend. Fast iteration is difficult, and operators must invest in significant hardware or hosting costs just to get started. As the DEX ages, these barriers will only grow.

Change Your Perspective, Change the Problem

Imagine a day five years after an Ambient instance launches. A major sporting event is driving high iFi transaction volumes. One by one, overloaded graphcache-go instances fail. Even with state snapshots, new instances require over an hour just to copy the data and start syncing. User experience and iFi applications suffer.

This is not the future we want to build. A key design goal for iFi DEX is to ensure the frontend and backend are deployable with full automation on a low-cost VM (under $15/month) in under 15 minutes.

Part of the reason we chose Ambient as our base is that its primary limitation—contract size— is something we could adjust within the Althea L1 EVM. However, to our surprise, that adjustment wasn’t necessary.

In the time since Ambient’s initial deployment, Solidity has advanced from v0.8.19 to v0.8.28. Simply upgrading to the latest compiler shrinks the main contract from 23.824KB → 23.376KB.

The real breakthrough came with Solidity’s new via-IR compiler optimization, which translates Solidity code into an intermediate representation (IR) before converting it into EVM bytecode.

Enabling via-IR freed an additional 2.236KB of contract space. The result? The same functionality, the same operations—now using just 21.558KB instead of 23.824KB.

This unexpectedly large reduction in contract size gave us more than enough room to add events and additional features without sacrificing performance.

Ultra-Fast, Event-Based Backend

With this newly available space, we added essential events to the iFi DEX codebase and built a pure Rust backend that mirrors the graphcache-go API. This creates a seamless drop-in replacement, ensuring compatibility with existing Ambient DEX tooling and frontends.

The iFi DEX backend is a monolithic, statically linked Rust application with an embedded RocksDB database. This means no local dependencies, and the same binary can be deployed across any Linux distribution—or even on Mac and Windows.

Even with basic event syncing, processing 10,000 blocks per second is trivial. With highly parallelized logic, the only constraints become network bandwidth and disk write speed.

Unlike graphcache-go, iFi DEX can sync using any full node, making high-throughput infrastructure far easier to build.

From a storage perspective, only the data necessary to serve queries is retained, leading to disk usage that is approximately 1000x lower than a graphcache-go subgraph.

Operationally, a full sync that previously took 10 days or more now completes in just 10 minutes—on dramatically lower-cost hardware.

Conclusion

The next generation of DEX software is about abstraction— allowing users to swap seamlessly as part of larger applications, whether cross-chain or within iFi on Althea L1.

Achieving this requires better tooling and reliability. Open access, combined with ease of deployment, fosters greater decentralization and better user outcomes. We’re excited that iFi DEX is pushing the boundaries here.

When we set out to replace the original Ambient backend for iFi DEX, we didn’t expect to find a practical solution for Ethereum mainnet. But we now anticipate Ambient deployments transitioning to event-based backends and contracts using the strategy we have pioneered here.

Next week, we’ll be diving into iFi DEX’s governance integration!

This topic is covered more concisely in the Althea whitepaper, but I wanted to take the opportunity to talk about the Althea L1 scaling philosophy and detail how it provides similar scaling performance as ZKP (zero knowledge proof) rollup based solutions while also being better suited to the iFi use case.

The ZKP tradeoff space

First, some ZKP context. For programmable transactions such as DeFi, ZKP (zero knowledge proof) technology provides an attractive scaling option, because programmable transactions can be difficult to execute and require significant permanent storage. A ZK proof of the same transaction has a fixed size and a fixed cost to verify which is significantly smaller than the average transaction, much less the largest transactions.

The downside is that computing the ZK proofs takes time, and the storage requirement is not so much reduced as it is moved around. Someone still has to store the original transaction information to understand what the ZKP validates.

I think this is a really undertold point around ZKP scalability. State growth, not immediate throughput, is the real problem that ZKP ‘solves’.

For Ethereum, every validator currently stores every transaction. Ethereum could easily operate with dramatically larger blocks with moderate impact to centralization, but over time the storage size increase would become fatal. As long as the state growth of a blockchain occurs at the same rate digital storage costs decrease, the cost of operating the chain stays the same. Once a blockchain has state growth that occurs faster than technological advancement in storage, the cost to operate will eventually become insurmountable.

Using a ZKP rollup strategy it is easy to ensure that the state growth of the L1 is very low. So low, in fact, that operating the chain would become cheaper and cheaper over time. Even better, since ‘rollups’ roll many transactions into a single ZKP state, growth on the roll-up chain is completely decoupled from the number and complexity of transactions. 

But now we come back to the inconvenient details: in this new ZKP paradigm the storage has to go somewhere. Transactions are no longer being stored by every validator, they now only store proofs. Now the user, or more likely some data availability layer, must store actual transaction data. Otherwise, the ZKP stored on the chain can be verified, but don’t provide any info about what has actually happened to your wallet in the past, or even your present value of tokens. The “Zero Knowledge" part of ZKP cuts both ways. Before ZKP, data could never be lost from an operational blockchain. So long as a new block could be produced, all data would be retained. But with ZKP rollips, the responsibility of retaining historical data has moved from everyone to a few parties, potentially even no one at all. Without a hard requirement to maintain historical data, chain history can be entirely lost over time.

This is where data availability layers come in as an answer to ?. These are conceptually similar to a Filecoin, except with the ZKP rollup chain itself as a sovereign customer. Node operators are then paid to store some subset of the chain state with some number of copies for redundancy.

A chain may choose to pay for only a year of state storage, leaving clients to pay for storing their own data themselves if they want more. Alternatively, the chain could trade indefinite storage availability for a better user experience at the cost of higher fees.From the perspective of a DeFi user, a ZKP rollup is not particularly lower cost, faster, or otherwise better to use. But from the perspective of the roll-up chain itself capacity is hugely increased.An apt comparison here may be with road expansion. Drivers are always convinced that one extra lane will reduce their commute times. But in reality the average commute time remains constant and the extra lane is consumed to support more people making the same commute in the same amount of time. The benefit of ZKP is ultimately not really user facing in the scaling sense. ZKP for privacy is a separate discussion.

The Althea L1 scaling design

CometBFT provides instant finality and decouples state growth from validation

The major theme in Althea L1’s design is the tension between programmable transactions and simple transfers for real world goods like bandwidth in the case of the Althea Protocol.

As discussed in the previous blog (link again here) DeFi transactions are much less sensitive to finality latency. If a DEX swap doesn’t finalize you don’t lose the input tokens for the trade. If a customer’s payment for a candy bar fails to finalize, they have already left with the product..

This makes the computational latency of a ZKP rollup much less attractive for Althea L1. iFi programs, specifically micro-transaction generating applications like the Althea Protocol, are limited by the latency of finality.No one wants to wait around several minutes for a prover to finish generating a proof of their candy bar purchase. Solutions to mitigate this latency in ZK would  involve a prover effectively promising to include your transaction in the next rollup proof.These promises do meet the speed and finality requirements for transfers of goods, but not the decentralized trust requirements. In order to generate a decentralized, guaranteed promise of inclusion at low latency you would effectively need a smaller blockchain just for returning inclusion promises.

In this case, Althea L1 replicates the most critical component of ZKP rollups for scalability, without sacrificing time to finality or trust. It does so through controlling state growth by allowing the chain to operate without the entire history.

Althea L1 uses CometBFT, a DPoS (Delegated Proof of Stake) consensus engine. A key concept of CometBFT is that in order for the chain to move forward, ⅔ of validator voting power must sign the next block. This means every block is instantly final, but comes with the consequence that it is relatively easy to halt a CometBFT-based chain. Compare this with alternatives likeETH2 consensus where the inactivity leak concept ensures that the chain automatically forks if finality fails.While the ETH2 solution may seem immediately more attractive, it’s this property of ETH2 consensus that really drives the need for keeping state growth on the main chain low. If it is possible for multiple forks to be valid, a node requires the entire history to determine which one to follow. 

For CometBFT chains, instant finality makes extremely compact proofs of chain state trivial. A proof must simply provide a history of validators entering and exiting the set (an infrequent operation in DPoS chains). Even for chains with a long history this proof is only a few kilobytes. The small size of these proofs is extremely important for bootstrapping light clients quickly and easily, a strong requirement for IoT, machine-to-machine payments,   and embedded applications. 

These compact light client proofs allow Althea L1 validators to operate without all historical blocks,opening up Althea L1 to utilize data availability layers in the same way a ZKP rollup chain would and meeting that same critical property where the minimum operational state grows more slowly than storage technology advances. 

Resolving transaction capacity

Now that the problem of storage growth is resolved, short-term throughput comes into focus as the next problem to tackle.While a ZKP rollup chain can process a theoretically infinite amount of transactions while moving around a fixed amount of data between the validators, Althea L1 validators still need to gossip about all constituent transactions.

We’ll discuss three solutions to this: first, a simple sanity check to evaluate the performance an ideal implementation should be able to achieve; second, load reduction strategies using payment channels; and finally, an exploration of Interchain Security, which is essentially Ethereum sharding made significantly easier by CometBFT’s instant finality and native IBC bridging.

Theoretical capacity of an ideal implementation

The minimum encoded size of an Althea L1 MicroTx is 137 bytes: 40 bytes for the addresses, 33 bytes for the signer's public key, and 64 bytes for signatures.

Assuming the limiting factor is bandwidth capacity between validators, and all validators have a  10Gbps connection, this gives a theoretical capacity of about 10 million MicroTx transactions per second. This estimate assumes that validators generate and agree on a 6.25GB block every 5 seconds.The current block creation and tx gossip code in CometBFT is nowhere near this efficient, but improvements over time will bring it closer to ideal theoretical capacity. 

It’s entirely possible that cheaply available server bandwidth will grow faster than the rate of block size increase. In the whitepaper we roughly estimate that 3 million TPS would be sufficient to cover all home internet users in the United States fully saturating a gigabit connection each.

Load reduction via payment size adjustments and using payment channels

Simple two party payment channels enshrined on chain with as a prioritized transaction type provide a viable solution here that not only increases capacity, but also reduces latency.Let’s use an Althea Protocol node as an example. As transaction throughput on the Althea L1 network becomes more congested with micro-transactions, the node can respond by reducing the frequency of its payments.

To do this, the node extends slightly more trust to the counterparty purchasing bandwidth, in exchange for paying fewer transaction fees. Specifically, it increases the settlement period—shifting from processing payments every few minutes to processing them every few hours.Since Althea L1 validators are incentivied to include MicroTx payments based on a fee percentage (link to whitepaper), rational actors will fill blocks with MicroTx payments in order of decreasing size and a client can simply observe the minimum fee included in the last block.Once this is no longer practical, the node can set up a two-party payment channel where funds are locked into the channel and the last channel signature can be submitted by either party to the chain at any time to pay out the channel balances.Presently, only an EVM implementation of payment channels is available, though translating this implementation to a native MicroTx module transaction can be done at any time chain capacity seems insufficient.

Combine these load reduction strategies with high bandwidth requirements for validator nodes and a very efficient gossip implementation and it starts to look eminently possible to serve the entire world's bandwidth needs with a single chain. 

Interchain Security

Using Interchain Security the main chain and its validator set effectively become a rollup chain, although one that uses CometBFT light client proofs instead of ZKP.

The sidechains would regularly post state proofs and cross sidechain transactions to the main Althea L1 chain. To ensure efficiency, blocks on the main chain would be kept extremely small, allowing payments made on the sidechain to finalize on the Althea L1 chain within just a few seconds.

Clients would be able to verify the inclusion of payments directed to them on any sidechain by maintaining only a light client for the main chain. As previously discussed, both the sidechains and the main chain could still leverage data availability services in this setup. This approach ensures that state growth on each individual sidechain remains highly manageable.

While the Interchain Security solution has all of the same scaling properties as a ZKP rollup it is significantly less power hungry, lower latency, and doesn’t require specialized hardware such as GPU or ASIC to settle quickly.

In general, we anticipate scaling optimizations, plus payment channels, will allow Althea L1 to operate at scale without having to add the complexity of an interchain security / sidechain design.

Hey, what about the EVM?

The MicroTx module was designed to be easy to scale, and transactions are a fixed small size. But what about the EVM?

While Althea L1’s MicroTx history can be safely discarded, the validators must have the entire EVM state in order to correctly compute the results of future EVM transactions.This is true, and part of why DeFi on Althea L1 is relatively expensive. The goal is to maintain EVM state growth at or below the level of storage media improvements and potentially implement a state sunset procedure.Sidechains could be used to design specific shards for EVM and DeFi operations, fees would then be slowly increased over time on each of these EVM shards to push users to a younger shard with lower fees. 

Conclusion

As someone with years of experience designing and building in the blockchain industry, this topic has been on my mind for quite a while—mainly because of its surprising conclusion. From a technical standpoint, blockchain scaling, at least for the capacity that Althea L1 might ever realistically need, is essentially a solved problem.

Moreso, it’s a solved problem without even having to reach for the most bleeding edge solutions. We have the luxury of tailoring the Althea L1 scaling design for lower latency and power usage by avoiding ZKP. 

This speaks to how much blockchain technology has improved over the last decade, and it also exemplifies an industry that has built superior technical solutions faster than it has secured wide adoption.

The Althea Protocol's design focus on implementing practical iFi, exemplifies an inflection point in blockchain technology, driving the utility and power of that technology to users and places where it was not before imagined

About Althea: Since 2018, Althea’s technology has brought blockchain-enabled internet access to thousands of homes and even the most challenging topographies. To read more, go to althea.net. 

To get involved, join Althea’s Discord: https://discord.gg/dT3sxP5gAw

Introducing Althea L1

Althea L1 is a groundbreaking blockchain designed to support Infrastructure Finance (iFi) a new category of blockchain applications that combines concepts from DeFi (Decentralized Finance), DePIN (Decentralized Physical Infrastructure), and RWA (on-chain Real World Assets) to deliver an end-to-end, on-chain application. This spans the entire value chain, from customer payments to the capital and financing of building infrastructure.

In 2017, Hawk Networks developed the Althea Protocol, which commoditizes bandwidth. It enables seamless, autonomous data purchasing by the byte from one or multiple providers with high reliability. The protocol facilitates programmatic and transparent transactions, revolutionizing how telecom is funded, built, and operated.

The innovative design of the Althea Protocol allows it to operate efficiently between machines in a network and at the scale of the entire internet. It’s not just an alternative form of customer billing but an entirely new approach to payment and settlement between internet service providers.

Why Althea L1 Requires a New Settlement Layer

This unique machine-to-machine billing system demands a radically different settlement layer. Comparing blockchain throughput to credit card settlement is a common exercise, as both measure capacity in “transactions per second.” However, the similarities end there. The definition of a transaction and the limitations of its execution differ significantly in the blockchain environment.

The goal of Althea L1 is to bridge the gap between real-world transactions and programmable DeFi transactions. It enables applications like the Althea Protocol to thrive by using an account abstraction called Liquid Infrastructure, which integrates seamlessly with both domains.

A Hybrid Approach for Two Distinct Transaction Types

Serving these two types of transactions on the same platform creates inherent tensions, as each has unique requirements and optimizations. These differences may not be immediately obvious but can prevent scalability if overlooked. By carefully designing an environment tailored to each use case, Althea L1 stands out as a platform capable of hosting both types of transactions effectively.

DeFi Transaction Properties

• Finality can be slow because internal consistency matters more than absolute certainty of execution.
• Failing transactions do not disrupt general operation.
• Fees can vary widely, with user activity adapting to fee changes.
• Parallelism is in tension with composability: the more parallel a programmable transaction, the fewer applications it can interact with simultaneously.

Real-World Transaction Properties

• Finality time is critical; payments must be final within a few seconds.
• Reliability is essential; fast finality must be consistently available.
• Fees must be predictable; user activity cannot adapt but must be facilitated. For example, a payment method failing while buying gas is unacceptable.
• Payments are easy to parallelize without losing functionality. Sequencing is only relevant in a minority of transactions.

How Althea L1 Balances These Needs

Althea L1 provides a hybrid environment where payments for real-world goods are processed quickly and efficiently while connecting to a programmable environment that can be dynamically throttled based on payment demand. This allows iFi applications like the Althea Protocol + Liquid Infrastructure to operate reliably for business use while remaining natively linked to DeFi capabilities.

As revenue from real-world transactions is fed into DeFi primitives like Liquid Infrastructure, layers of composable DeFi can be seamlessly integrated without disrupting customer transactions for bandwidth or other off-chain goods.

Dynamic Blockspace Allocation

Althea L1 assigns blockspace differently based on transaction type, creating an incentive structure that prioritizes the reliability of microtransactions used in utilities and telecom while balancing DeFi usage dynamically.

For example, separating Althea Protocol transactions from other on-chain traffic optimizes throughput and reliability for MicroTx transaction types. If Althea L1 faces high transaction demand, the protocol can raise the minimum size of a MicroTx transaction. This adjustment compels Althea Protocol clients to increase their payment threshold, ensuring the blockchain remains functional and includes Althea Protocol transactions in a timely manner.

Conclusion

Althea L1 is a pioneering blockchain that bridges real-world transactions and programmable DeFi, enabling applications like the Althea Protocol to thrive. Its hybrid design prioritizes reliability for microtransactions while supporting scalable DeFi applications, seamlessly integrating on-chain programmability with real-world practicality. By dynamically adapting to transaction demands, Althea L1 sets a new standard for blockchain-enabled infrastructure—purpose-built for utilities and infrastructure—transforming how networks are built and operated.

For more details about Althea L1, the Althea Protocol, and Liquid Infrastructure, refer to the whitepaper.

This is the first in a series of blog posts about Althea L1 and its features. Stay tuned for more updates here soon.

About Althea: Since 2018, Althea’s technology has brought blockchain-enabled internet access to thousands of homes and even the most challenging topographies. To read more, go to althea.net. 

To get involved, join Althea’s Discord: https://discord.gg/dT3sxP5gAw

By enabling access to licensed spectrum in flexible ways, the partnership addresses the growing need for reliable, high-capacity wireless solutions while lowering the historical barriers to entry. Once reserved for large telecom entities due to cost and complexity, licensed spectrum is now available to a much broader range of participants—making it possible for small operators, local communities, and businesses to build telecom networks that provide high performance mobile, IoT, or broadband solutions. 

Democratizing Telecom Infrastructure

This partnership signals a critical step toward realizing Althea’s vision of an open, decentralized platform that redefines how telecom networks are built, funded, and operated. Traditionally, building telecom networks required centralized coordination, costly infrastructure investments, and exclusive access to key resources like licensed spectrum. For smaller or community-driven initiatives, these hurdles were nearly insurmountable.

Now, through Althea’s collaboration with Globalstar, Global Telecom, and Alliance the barriers are significantly lowered. Access to Globalstar’s Band 53 spectrum and Althea’s KeyLTE/5G allows entities of any size to take advantage of the reliability and efficiency of licensed spectrum, a critical resource that was once out of reach. The spectrum can be used by itself, or aggregated with CBRS for higher performance. This democratization is not just a breakthrough for smaller networks—it’s a transformative shift in how telecom infrastructure can be deployed and operated across a wide variety of industries.

The Althea Platform: Decentralization Meets Dynamic, Scalable Networks

At the core of this innovation is Althea. Unlike traditional telecom models, which are controlled by centralized entities and involve complex, hierarchical management structures, the Althea blockchain-based ecosystem with its dynamic routing and machine-to-machine based payments. invites multiple participants—whether they are large telecom operators, local municipalities, or grassroots ISPs—to leverage its core protocols to build dynamic, scalable networks.

Althea automates many of the traditionally labor-intensive processes in network management, such as payment settlements, routing, and bandwidth allocation, through smart contracts. This automation dramatically reduces operational overhead, enabling a more efficient use of resources and making it easier for smaller operators to participate. As a result, network operators can focus on delivering high-quality services without the administrative burden that often comes with legacy telecom systems.

Unlocking the Power of Licensed Spectrum with KeyLTE/5G

One of the key advantages of this partnership is how Althea unlocks the power of licensed spectrum leveraging the KeyLTE/5G solution. By integrating Globalstar’s Band 53 and CBRS Band 48, Althea enables network operators to build high-capacity LTE and 5G networks without the congestion and limitations that come with unlicensed spectrum.

Historically, smaller or decentralized networks were unable to access licensed spectrum due to its high costs and the complexities involved in securing leases from large carriers. With this partnership, entities of various sizes can now access a comprehensive, off-the-shelf solution that includes the necessary radio equipment, core network components, and licensed spectrum. This streamlined offering provides an efficient way to deploy mission-critical applications—whether it’s delivering broadband to remote areas, supporting industrial IoT systems, or enhancing smart agriculture—with reliable, high-performance connectivity.

Building Flexible and Inclusive Networks

One of the most compelling aspects of the Althea platform is its flexibility, enabling participants to build telecom networks tailored to their specific needs. This “right-sizing” approach, combined with off-the-shelf tools, makes deploying broadband or IoT infrastructure both seamless and cost-effective.

For instance, a farmer looking to connect cameras, monitoring devices, and smart sensors could build an LTE network at a cost similar to setting up a Wi-Fi network. The advantage is that LTE offers superior reliability and flexibility, but now with the same affordability and ease of deployment as Wi-Fi.

Althea’s open, decentralized framework—featuring tools, smart contracts, and protocols—empowers a broad range of entities, from local ISPs to industrial IoT operators, to build and manage their telecom infrastructure. This inclusivity drives innovation and competition, ensuring no single entity dominates access to network resources or spectrum.

Blockchain-Powered Payments and Real-Time Network Management

Another key aspect of Althea’s innovation is its blockchain-powered payment system. Traditional telecom billing models rely on rigid cycles, often charging customers for services they may not fully use or need This has created unnecessary rigidity and precluded access to nearly 1 in 4 Americans, and stymied the growth in the IoT market. . By enabling real-time micro-transactions operators have flexible choices for capturing value, and users have choice and fair value.   Users offers a flexible, more personalized experience for customers, who can adjust their service levels on-demand without the constraints of fixed contracts or pricing plans.

Additionally, Althea’s blockchain architecture enhances the efficiency of network management. Its smart contracts make decentralized, automated decisions on routing and bandwidth allocation, optimizing network performance in real time. This decentralized routing model ensures resilience and opens up exciting markets in IoT, even in challenging environments where traditional networks might falter.

Harnessing the Power of Real-World Assets (RWA) in Telecom

Through blockchain integration,  real-world assets like telecom towers, fiber optic cables, or even spectrum, can become tradable and programmable, unlocking new ways to utilize and earn from previously siloed infrastructure. This approach invites participants  to join in building the future of decentralized infrastructure across industries, from telecom to utilities, enabling the use of RWAs as a critical asset class in emerging decentralized markets, and creates flexible ways to coordinate both funding and operation of infrastructure. This partnership shows how telecom companies are evolving to embrace more flexible structures for assets, including licensed spectrum assets. 

With Althea’s Layer 1 (L1) blockchain, physical assets are seamlessly connected with decentralized network operations. This makes it easier for more participants to own, operate, and earn from telecom assets ( RWAs,) allowing them to become active stakeholders in physical infrastructure that was once the domain of large entities. This decentralized approach ensures that real-world telecom assets—like spectrum and network equipment—can be traded or leveraged efficiently, empowering even smaller operators and communities to participate 

Althea’s Liquid Infrastructure model also creates a real-time marketplace for essential real-world assets for modern telecom systems. This marketplace allows operators to dynamically buy, sell, and manage these resources based on immediate needs. The ability to trade RWAs such as telecom towers, fiberoptic cables, and spectrum in real-time helps to create a more efficient, adaptive, and liquid telecom ecosystem. Communities can scale their networks organically, adding capacity as demand increases without the need for large upfront capital investments, making telecom infrastructure more accessible and efficient for all. Althea’s integration of RWAs positions it as a leader in transforming physical assets into dynamic, tradable resources, contributing to the growing adoption of tokenized RWAs in the decentralized economy.

A New Era for Connectivity

With the rise of the machine economy, AI, and a future filled with seamlessly connected devices, the need for a new paradigm that democratizes access to infrastructure has never been more critical. By combining blockchain-powered network management with access to licensed spectrum, Althea is paving the way for a more inclusive telecom landscape. High-performance telecom solutions, once limited to traditional broadband providers, are now within reach for organizations of all sizes, empowering them to thrive in emerging markets like connected machines and IoT.

As telecom continues to evolve, this collaboration between Hawk Networks, Globalstar, Global Telecom, and Alliance serves as a blueprint for the future—one where technology is flexible, inclusive, and shaped by the needs of its users rather than the constraints of legacy systems. With this partnership, Althea is not just driving change; it’s crafting the blueprint for the telecom networks of tomorrow.

About Althea: Since 2018, Althea’s technology has brought blockchain-enabled internet access to thousands of homes and even the most challenging topographies. To read more, go to althea.net. 

To get involved, join Althea’s Discord: https://discord.gg/3w2ZPKue

In order to maintain formatting please view the PDF version of the whitepaper linked above.

Althea's pay-as-you-go system is a unique billing model that allows users to pay for the internet services they consume on a usage basis rather than commit to a fixed monthly subscription. This approach to billing can be particularly helpful for households who struggle to pay the same fixed bill month after month. Customers can use services as needed without long-term commitments or contracts, and they can adjust their usage and spending based on their financial situation.

Althea is a pioneer in telecom, leveraging blockchain technology and a mesh network to provide reliable and affordable internet connectivity. Their network selects the cheapest, best connection available on a per-second basis, improving both reliability and redundancy for customers. Althea's approach means that customers will get low latency and no data cap, with the average monthly bill only costing between $30 and $40. And if customers enroll in the FCC’s Affordable Connectivity Program (ACP), their service can be free.

Customers who refer a friend to Althea will receive a one-time $50 credit towards their account and can receive up to a $250 credit if they have referred five or more friends.

Las Cruces residents interested in Althea services can go to hawknetworks.net/las-cruces and fill out the contact form.

About Hawk Networks Inc.

Founded in 2018, Hawk Networks (Althea) designs and deploys modular, open-access broadband internet and connectivity solutions, bringing fast, reliable connectivity to even the most remote and challenging environments.

Althea is a holistic system for building networks with a unique set of “broadband legos” to build networks that work with communities, instead of a top-down solution that doesn’t fit for many indigenous networks. KeyLTE further unlocks connectivity solutions beyond broadband to the home which supports many different applications including cellular, smart and precision ag, sensors, monitoring, and smart cities applications.

Based in British Columbia, Four Our Future is an Indigenous-owned corporation founded by Indigenous leaders across Canada is very familiar with supporting digital equity and bridging the digital divide in a resilient and sustainable way.  Four Our Future specializes in developing and implementing sustainably-focused strategic plans, business plans and community based economic development models. Their team identifies and addresses the dichotomy between Western-corporate business models and more ‘holistic’ Indigenous/community development models, drawing from their collective experiences living, working and learning from over 300 of Canada’s 630+ Indigenous governments.

The Unlock Connectivity Grant will provide Four Our Future with no-cost network design and support services, as well as $15,000 towards LTE hardware.  One of the first projects that Four Our Future will embark on is a smart and connected building in Vancouver BC that will serve as a testing and educational center for broadband capacity building. This unique hands-on approach will give indigenous leadership and IT professionals the opportunity to see a network build out in real time and envision the connectivity possibilities in their communities.

Four Our Future Indigenous Economics is very pleased to accept this award and the new relationships with Hawk Networks, Althea and all the technical teams responsible for the opportunity to bring greater prosperity to First Nations communities.

Education and capacity building is one of the most important and overlooked parts of building community networks. These components enable anyone to use technology to solve the digital divide. This knowledge shouldn’t be siloed in big tech or academia, and the Four Our Future <> Althea partnership aided by the Unlock Connectivity award will make tangible steps towards the goal of democratizing access to this technology to the more than 600 First Nations in Canada.

The partnership will also empower web3 educational opportunities, empowering youth to enter into this emerging economic  opportunity in a tangible way. Althea’s interoperable micro-transaction and composable billing solutions intersect with Four Our Future’s work in NFTs (400 Drums) to provide access for anyone to learn about how blockchain technology can be used in real world applications.

400 Drums is a #web3 campaign dedicated to supporting Indigenous Elders, artists, language speakers and tour operators in harnessing new online platforms to monetize their skills while sharing their teachings and stories. Drums are handcrafted by David Fierro of the Okanagan Nation in collaboration with Tamara Goddard of the Saulteau Nation. Once done, the instruments are photographed and sold as NFTs.

We are excited to begin this important work with the Four Our Future team, and look forward to breaking ground on the first connected building in early 2023!

About Hawk Networks:

Hawk Networks develops the technology underpinning the Althea system including:

• Price Aware and Software Defined Routing
• Peer-to-Peer Micro-transaction platform
• Gravity Bridge, a decentralized blockchain bridge
• KeyLTE, an embedded home router core
• Network Operator Tools and Managed Router Firmware
  

The Unlock Connectivity Grant will start accepting new submissions in early 2023….stay tuned!

Althea is a platform for distributed payments in a cooperative or multi-entity network. It combines a micro-transaction framework for programmatic and near real-time payments between peers. Althea network offers a powerful vision for the future by removing the friction of legacy structures and unlocking access to finance and connectivity for all. It is a solution for the inflexible payment methods and many capital inefficiencies that prohibit infrastructure investment.

The internet is the essence of how we communicate, how we access interaction, speak our truths, conduct business and explore opportunities. It's where we learn, associate with others, and where the opportunities of the future are. So, it’s no surprise that a core use case of the Althea platform is connectivity.

The legacy structures of connectivity and the markets for last mile internet, showcase the inefficiencies common to legacy models. They are capital inefficient, the process creates friction and long sales cycles. The reasons that ISPs have some of the worst customer satisfaction scores isn’t because they are evil and malignant, but because the systems are fundamentally misaligned.

The current models don’t work well for last mile providers either. Take New York City for example, where roughly 29% of households lack access to broadband. That’s a market of over 3,000,000 people that legacy telecom does not serve.

In the core of the internet, however, where carriers charge each other, we create instances where markets are open and packets flow freely over vast interconnected networks. Costs go down, capacity goes up. It works.

Althea is the thesis that we can bring this open marketplace to the user and shift the balance of power, ownership and choice, with systems that work well and align user and provider.

Althea Platform Overview

Althea provides software defined networking in conjunction with tokenized “liquid” infrastructure to create fully democratized, open market network architectures

• A powerful blockchain based platform for micro-transactions, settles payments immediately, programmatically and transparently.
  
• Althea’s software defined networking algorithms, built into its routers and devices, allow users to select desired cost and performance parameters and the network dynamically routes traffic to meet those requirements
  
• Tokenizing the infrastructure (Liquid Infrastructure) allows anyone (operators, end users, funds, individuals) to participate in the funding and ownership of the physical assets in the network
  
• Althea democratizes networks by enabling buyers of any scale to participate in the ownership and economic benefit of network assets – including new assets that potentially create competitive infrastructure or access underserved markets

Under the Hood: The Althea Tech Stack

• Price Aware Routing and micro-transaction billing
• User Focused Experience - Router dash and embedded wallet, plug-and-play implementation
• Network operators: Network support and operator tools, managed routers and transparent metrics
• Hardware Agnostic: Fixed wireless, fiber, NOW KeyLTE
• Blockchain Interoperability: Gravity Bridge and Interchain Accounts
  
  

Pay Per Forward: Every router loaded with Althea software contains our unique forwarding protocol, which pays network routers for forwarding service. Creates near real-time settlement of transactions, with stable currencies, programmatically and transparently on the blockchain.

Price Aware Routing: Althea utilizes a dynamic, distance vector routing protocol weighted by user choice of cost or latency. This agile routing system enables resiliency, reliability and modularity.
By default, Althea routing protocols will try to select the route with a balance in quality and cost. From the Althea Router Dashboard, individual users may set their service to prioritize quality and price on a sliding scale.

Operator Tools: Althea’s Operator Tools is an online dashboard with a suite of tools for monitoring, managing, and deploying networks based on the Althea platform. Operator Tools provides a place to keep track of both the technical and financial aspects of your network.

Gravity Bridge: Gravity Bridge is decentralized blockchain, bridging Ethereum and Cosmos ecosystems.  Connected to Althea, Gravity enables distributed bandwidth networks to use stablecoins on Althea’s sovereign chain, ensuring telecom level reliability and seamless UX.

KeyLTE: Althea’s KeyLTE revolutionizes the deployment of public and private LTE networks by creating a simple and flexible toolkit for LTE. Easy plug-and-play design and maintenance allows builders to scale their networks to meet their needs and aspirations.

Holistic Network Implementation

This modularity of design becomes the building blocks of holistic networks. Pay-for-forward billing enables a variety of different investment structures, while the service can be maintained sustainably locally. The distributed, capital efficient design and flexible payment implementation opens new markets that were previously siloed behind rigid rental structures.

The network operator support software and plug-and-play design empowers more entities to build and encourages new entrants to the market, blending the distinction of public and private networks.

With KeyLTE and pay-per-forward revenue share, networks can be built and extended nearly limitlessly and used for mobile and industrial applications, including precision agriculture, smart cities, iot and broadband.

User Driven Design

Althea was designed to provide access to the composability and modularity of blockchain systems  with an intuitive user interface. Similar to how users can interact with browsers without the educational lift of understanding TCP/IP, Althea created an embedded, easy to use wallet that doesn’t require any special knowledge or expertise.

In the future, this wallet will connect with the Gravity Bridge, an interoperable blockchain bridge between ethereum and the Cosmos ecosystem. Together with Interchain accounts and single transaction signing, Althea users could access other blockchains, DEXs, and utilities all from their dashboard.

Currently, users pay in stable digital currency “digital dollars” which is distributed to entities with infrastructure in the network or who maintain the service components. This creates an equitable, transparent, and programmatic coordination layer for distributing network revenue.  An easy-to-use payment widget allows users to load routers using a debit card, and available customer service provides all expected functions of a legacy ISP, with the added benefits of a decentralized network, instead of a monopolistic one.

The Future of the Platform

Althea was founded in 2018 and since its inception, networks using the platform have been built across the US in 8 states, as well as Africa, proving that better systems can not only open new markets in challenging areas, but can flourish.

With the upcoming launch of Althea’s blockchain for telecom, the platform will migrate from it’s home on xdai to Althea chain and connect to the Gravity Bridge, opening up new opportunities for growth and welcoming new ecosystem participants.

Althea is a paradigm shift from brittle and inefficient legacy systems, to open and accessible structures, that unlock the future of finance and connectivity.

Application window is open 8/1/2022 through 10/01/2022 @ 5pm PDT

Application link: https://forms.gle/9mmyuAWgaNjvL1Rm9

Who can apply: New and existing network operators including municipalities, cooperatives, tribal, non-profit, and other organizational structures that operate a network.

Award: up to $15,000 of LTE Equipment to build an Althea KeyLTE deployment.  Design and planning support for the build would be included with impact on your proposed grant budget.

Impact: Provide a unique and/or creative solution use case for the use of Althea’s flexible KeyLTE core that meets a connectivity pain point. Show a pathway to a heterogeneous network, serving broadband to fixed locations, as well as IoT or smart (anything).  Whether it’s a farming community who’s looking to expand smart and precision ag or a rural area looking to provide connectivity to EV charging stations and remote wildfire cameras, your unique challenges are what we are looking to assist you to solve.

What is KeyLTE:  Althea is a platform for solving the "last mile" problem for broadband internet.  Hawk Networks Inc. derived the IP for a medium-agnostic revenue share and SDN routing, with a resilient and flexible blockchain settlement layer.  The concept we are submitting for this challenge is a solution for building plug-and-play, heterogeneous LTE networks that are capital efficient and agile, empowering any size entity to build and maintain a sovereign network. Althea technology is in production in six US states, Puerto Rico, and Africa and we are iterating on the authentication process for 5G networks.

KeyLTE revolutionizes the deployment of public and private LTE networks by creating a simple and flexible toolkit for network design and deployment.  Instead of expensive, unwieldy legacy core topologies, KeyLTE networks can be built easily and affordably in a variety of environments. KeyLTE is agile and lightweight enough to embed the EPC (Evolved Packet Core) in the Althea smart, home router (with an average cost of $200) and flexible enough to scale to nationwide deployments.

By embedding the EPC into the home Althea router instance, LTE networks can be deployed as inexpensively as you would install a home user, or even convert a subscriber or home deployment into an edge core with a simple firmware upgrade. This minimizes power consumption and footprint, further reducing capital outlay.  The KeyLTE router needs minimal configuration and can be maintained remotely in the Althea Operator Tools dashboard.

Network operators and private enterprise can utilize system integration and support from Hawk Networks, or utilize the open access framework and decoupled customer service layer, to partner and revenue share with other entities. This supports network builds from entities of many different sizes and capabilities, and empowers private enterprise to leverage existing assets to participate in offsetting their connectivity costs. From the construction site to rural fixed wireless, KeyLTE and Althea’s dynamic mesh routing empowers self-healing, agile, and resilient topologies all without manual configuration.

KeyLTE can be deployed on the edge, extending out a fixed wireless network, or as a flexible private LTE core. Read More about Althea and KeyLTE with the links below.

Althea

Social Media

Documents

Case Study

In 2020, that’s exactly what Thomas Spellman needed for his rural community in Nevada County, nestled in the Northern California mountains. Nevada City is a small town of roughly 3,500 people with hills, valleys and tall pine trees.

The Challenge:

Build a broadband network for a community of 3,500 people with very few alternatives.

Initial Capital: $10,000

Terrain: Rugged, tall trees, hills & valleys

The Solution:

Althea Platform

Medium: Fixed wireless solutions from Ubiquiti

Network Operator: Yuba Village Internet LLC, d.b.a. TreeLink, a private small business

Users and Relays: 146 total users, with 26 Relays

Althea is medium agnostic, and Thomas chose a fixed wireless solution for the initial build out, which provided flexibility and allowed for resilient and redundant failover as well as an agile build to the outlying areas of the network. In a legacy network, central towers would provide one point of coverage to the area. These towers are often expensive and don’t provide adequate coverage in challenging topology, like rural Nevada county.

With Althea, many homes, farms and businesses can become relays (hosted broadcasting antennas) and Althea’s payment system pays them for contributing to the network.  Similar to a distributed solar system, which pays users for providing solar energy back on the grid by measuring and remitting credit, Althea’s smart and price aware routing protocol, measures the amount of data forwarded through the relay antenna and compensates the relay host automatically in micro-transactions. Some relays in the TreeLink network are able to offset their internet costs and even earn extra money.

This relay system made it possible to build into hard to reach areas, and easy for the network operator, TreeLink, to negotiate roof leases without having to worry about sending payments to each relay, or manually calculate profit shares.

This also added flexibility to the capital needed to build out the network. Because the infrastructure of the network’s billing is separate from the customer service and governance layers, many unique funding mechanisms can be used.  Each user and relay pays for and owns the equipment needed to be part of the network, so a large initial investment wasn’t needed; the network grew organically, according to the needs of the community. When ATT DSL began shutting down older connections, TreeLink was there and able to rapidly expand to meet the needs of the community.

Thomas did have a tech background, but not experience with building ISPs, so he was looking for an easy-to-use solution that met the needs of the rural community, both from a technological and economical perspective.

I chose Althea mainly for the integrated metering and micropayment billing, as well as the smart throttling and secure VPN.  These features in every Althea router made it easy to build a mesh network of neighbors helping neighbors.  Another primary reason is because it's open source, which allows me to fix problems if needed, but the dev team has been great at continually evolving the system.  Last but not least, Althea's customer support has made it possible for me to focus on maintaining the network without needing to hire extra support staff. - Thomas Spellman, TreeLink

The Althea platform made it easy to not only build a network in this agile way, but also to monitor and maintain a diverse set of networking hardware from one platform. From the Althea Operator Tools dashboard, the network operator can monitor a variety of metrics related to network health, equipment temperature, link performance,and perform troubleshooting and maintenance. Because Thomas’ network is maintained by just him and another technician, it was important that the platform selected was easy to learn and also made it possible to operate and fix the network remotely.

As the network expands, Thomas plans to work with a community fiber network, Northern Sierra Broadband, and the open access Althea infrastructure will interconnect. He’ll also be deploying CBRS in the non-line-of-sight areas.

2021 saw new Althea networks in remote areas like Warner Valley in southeastern Oregon or the hillsides of northeastern California. These areas are representative examples, where centralized topologies failed to be capex efficient and as a result, these communities were left behind. Because of Althea’s platform, this is no longer the case.

Althea’s decentralized structure enables users to participate in hosting infrastructure and earning revenue, leveraging a community’s existing assets, and making it possible to build to the furthest frontier. The Warner Valley network is a particularly exciting example, as Althea serves the business, farms and even the emergency management and ambulance services across a 40 mile region of canyons and hills.

Let’s look back on the events of 2021:

2021 Recap

Partnerships and Grants

• NetEquity and Althea partnered to finance, build and operate last-mile broadband networks. Our first project will be in rural Nebraska where we will leverage the fiber optic network owned by OPTK Networks (“OPTK”). Utilizing Althea’s protocol to connect middle and last mile networks opens up new market opportunities for both the last and middle mile operators.
• Althea and George Mason University partnered with People Centered Internet and received a grant for the build out of a Tribal Resource Center, supporting Tribal Broadband Networks with technical expertise, particularly in the deployment of LTE networks.

• Althea partner network, Wave 7 Communications, in Enfield North Carolina was awarded the Epic Grant to further the growth of the network there.
• Althea announced a strategic partnership with Mohuman, a nonprofit focused on digital equity,  to build networks in underserved urban areas, starting in San Diego, California.
• Althea announced a new partnership with 5G zone contributor, Hub Advanced Networks, to build and expand new broadband networks and support telecom innovation as part of the Puerto Rico 5G Zone & Blockchain Ignition Lab initiative.
• Althea announced the launch of AgFiber and its partnership with Silo Communications to build out the first Althea fiber network in rural Illinois.
• Althea acquired a fixed wireless network in Washington state, which will be  an excellent opportunity to explore the tangible benefits of transitioning from a legacy to the Althea protocol  - announcement coming soon!
  
  

…And Product Launches!

• Operator tools features, including port liveness and LTE EPC monitoring, empower a vendor agnostic platform for network operators. A key advantage of Althea’s platform is the decoupling and incentivization of the customer service layers, systemizing not only the infrastructure build out, but also the coordination and support of broadband networks.
• Althea AgFiber combines a distributed fiber infrastructure with LTE to grow networks organically and expand the capabilities to the device level. Althea is an agnostic metering and billing platform, flexible across any medium and throughput, whether it’s 10GBps fiber networks or fixed wireless, the Althea protocol makes any network more cost efficient, agile, and opens new markets in traditionally underserviced areas.
• KeyLTE is the world’s first integrated home router LTE core, making expansion of IOT, private LTE and mobility networks plug and play. By embedding Althea’s KeyLTE core technology into our Smart Router, we've created the only home router in the world that gives you full ownership over your internet and full autonomy from traditional ISPs.

• Gravity Bridge Together with an amazing group of validators and community members, Althea supported the launch of the Cosmos® Gravity Bridge™, a decentralized, neutral blockchain, bridging assets between the Ethereum and Cosmos ecosystems. Althea users pay for their internet using stable coins, so a bridge between the ethereum ecosystem (with its proven stable coins), and the Cosmos ecosystem was a needed pillar of the Althea ecosystem. After almost two years of development and over 70,000 lines of code, this important component of the system is now live and operational as an independent, purpose-built blockchain–providing precisely this needed functionality. The Gravity Bridge is an innovative and decentralized approach to blockchain bridges. In 2022 it will continue to thrive as an independent ecosystem, supporting not only Althea, but also many other blockchains and DEXs in the Cosmos ecosystem.

With the foundational work on the systemization of Althea deployments, firmware components, and blockchain infrastructure completed, we are ready for the next chapter of Althea expansion in 2022.

The Year Ahead

The Althea Blockchain Launch

Althea users require a blockchain that is purpose-built to deliver the constant uptime and reliability needed for broadband networks. The Althea blockchain will be built for the many micro transactions of distributed telco, prioritizing quick and reliable settlement, unlike other blockchains that exist for financial transactions, where variable fees and uncertain settlement times are commonplace. Built with the Cosmos SDK, the Althea token will secure this settlement layer and Althea token holders will earn a small fee for each transaction, thus aligning the interests of all participants with the healthy growth and sustainability of the network.

This underlying blockchain infrastructure not only supports the metering and billing aspects of Althea’s networks, but also enables potential innovations in investment in long term fiber and vertical asset infrastructure. We expect some early applications on the Althea blockchain to explore new models of lending and funding infrastructure with fiber infrastructure, that can have recurring revenue for decades. This is another way that the Althea platform mitigates the demand risk of building networks, and aligns the interests of participants in the network.

Althea is also a portal for everyday users to benefit from seamless blockchain onboarding and access to decentralized financial tools. According to a recent FDIC study, roughly 6% or 14 million people in the US are unbanked, and a great deal more are under-banked. In fact, one of the most unanticipated aspects of our network roll outs was the surprisingly high contingent of users that relied on local corner stores or gas stations to access financial services.

Althea can address this need as well, by providing permissionless access to the empowering tools of decentralized finance. We will be looking for opportunities to gradually introduce these features to Althea users as the networks grow.

It is through this agnostic and open lens that Althea sees the future of internet infrastructure, and we have taken the path of supporting any medium of packet transmission, whether through fiber, LTE, fixed wireless, or even CanTenna! It was this hardware flexibility that enabled us to bootstrap through much of 2020 and to open markets where legacy models could not. While many of our early networks had focused on fixed wireless networks, Althea sees the future in dense fiber networks and LTE* at the edges.

What does that future look like?

Imagine stepping off a plane in a new state, and instead of your phone locating a compatible network signal negotiated by deals and roaming agreements, you will configure your choice of cost and latency and Althea’s price aware routing protocol will automatically connect to the best tower and cost automatically. The municipality you are visiting hosts infrastructure with 5G LTE antennas and they earn a portion of the revenue generated from your connection as well.

You jump into your Althea empowered electric vehicle with your synced Althea wallet. While idling and waiting for your passenger, your vehicle’s hotspot provides connectivity to nearby phones and IOT devices, earning you and the auto manufacturer revenue automatically. As you drive along, your phone and car connect to different network infrastructure and pay-as-you-use the infrastructure, your priority and tower connection set by your choices of cost and latency.

Once home, your Althea home router blinks cheerfully - the LEDs indicating you earned revenue by leveraging the balance you keep in the embedded blockchain wallet. On your roof are LTE antennas that also provide coverage in your neighborhood. You earn passive revenue, and your local network operators provide support and answer the 3 am phone call, should anything need repair or maintenance.

This is the Althea future - the foundations of LTE and fiber, blockchain and interoperability laid in 2021, now systemized and coming to fruition starting in 2022.

2022 Roadmap

We have seven new markets planned in 2022, the Althea blockchain launch is imminent and some exciting new partnership announcements are coming soon.

Q1 2022

• Launch of Althea blockchain!
• Althea networks live in Centralia and Tacoma Washington
• KeyLTE routers and available, expanding private LTE and fixed wireless use cases
• Development of LTE capable IOT devices
• KeyLTE pilots of Smart Ag integrations in Illinois, Oregon and Indiana
  

Q2 2022

• Introduction of new smart contracts for infrastructure investment
• New fiber network build outs
• Early tests of Public/Private LTE networks
• Expansion of LTE use cases
• Launches in Atlanta and Denver and rural Nebraska
• Puerto Rico networks
  

Q3 2022

• Integration of Althea blockchain in routers with a more seamless fiat on and off ramps
• MVP of integrated Defi with Althea routers
• Launches in San Diego, Puerto Rico and expansion in rural Oregon
  
  

Q4 2022

• Production public/private LTE instances
• Plug and play LTE for Smart Ag and Private LTE
• New Market launches possible in Carson City, rural Georgia and North Carolina.
• Support for partner networks and systemization of network protocol transition with existing networks
  
  

The current legacy models don’t work for either the shareholders (large telco companies shoulder huge amounts of debt and incur a high demand risk) or anyone that uses the internet. Siloing users into a monthly “rental” model encourages unnatural dead ends to network growth and the under-utilization of bandwidth and infrastructure. Althea opens up these markets, and gives back ownership, sovereignty and choice to the people who rely on them, much the same way that iTunes and YouTube revolutionized how we consume and purchase media content.

I was drawn to Althea for the importance of the key freedoms we have online and the impact that access to the internet brings. It is singularly powerful in it’s significance to a more equitable and thriving future. I have been proud to work alongside some amazing people towards this goal and I look forward to the continuing journey in the coming year.

Website

Althea FAQ

Discord

Twitter

*  LTE vs “5G” - Why do we talk about LTE instead of the usual “5G” that is used in most marketing campaigns?

You may have noticed we talk a lot about Althea’s LTE stack instead of putting “5G” or 5th Generation, front and center. There are two reasons for this deliberate nomenclature. First of all, 5G as a 5th Generation LTE protocol does bring interesting and exciting improvements, and Althea will be capable of utilizing those and supporting these 5th generation capabilities, but Althea’s agnostic protocol is also expansive and agnostic to generation past or present. Secondly, 5G has become such an overused term as to become relatively meaningless. Althea has a strong engineering and technical approach to building better systems, and we believe that accurate nomenclature with more specificity is a better way to operate.

*COSMOS is the registered trademark of the Interchain Foundation, and is being used by permission. GRAVITY BRIDGE is the trademark of Gravity Bridge.

This collaboration focuses not only on improving the resilience of Puerto Rico’s communications infrastructure, but also pioneering innovative approaches to internet access, smart Ag, and LTE networks. This 5G and blockchain research and development initiative will improve the quality of life for the people and businesses of Puerto Rico and extend coverage and affordable opportunities to even the most remote and challenging areas of the island.

Althea Network’s unique plug-and-play ISP platform utilizes blockchain technology to decouple the service and infrastructure layers of internet delivery, empowering end users to revenue-share by hosting network infrastructure. This is all accomplished in a user-friendly device that requires little to no training for blockchain novices to manage, thereby demystifying blockchain technology.

Powered by Althea’s Smart Routers and connected to Hub Networks’ vast fiber resources, last mile networks can be built in a flexible and agile way, delivering high broadband speeds and low latency. What’s more, users can choose to earn revenue by adding infrastructure to expand network coverage further. Each Althea Smart Router also features an easy to use blockchain wallet and firmware to meter usage and billing, allowing for the rapid and cost-efficient deployment of large, fast, and economically accessible broadband networks.

Althea is a leader in open telecom innovation and recently announced the introduction of the world’s first integrated LTE core. By embedding the LTE core directly in the home router, KeyLTE creates a small cell anywhere and enables the rapid and seamless deployment of LTE networks for data, mobile devices and IOT management.

“Access to connectivity and broadband is essential to business development, education and daily life ,” says Deborah Simpier, CEO and co-founder of Althea. “Althea and Hub Advanced Networks are committed to connecting more people through our work together in Puerto Rico”

“Althea’s wireless infrastructure expands the geographic reach of our broadband networks, but just as importantly, our unique, no contract billing protocol expands the demographic of people who will have access to fast and affordable broadband.”

“Althea is excited to work with Hub Advanced Networks and proud to contribute to the success of the Puerto Rico 5G Zone & Blockchain Ignition Lab.”

About Althea

Althea is a new way to access broadband internet that enables growth in the network through a unique system of revenue sharing. Althea was founded in 2017 with the vision to bring the open market and interoperable peering from the data center to the field and empower communities to build multi-stakeholder networks faster and more affordably than legacy telecom models.

Kenneth Gal: kgal@shinyobjectsinc.com

######. ######.   #####

About Hub Advanced Networks

Hub Advanced Networks is an Advanced Telecommunications and Technology company committed to enabling business leaders with the digital infrastructure and reliable connectivity needed to bring innovative services to market in Puerto Rico and the Caribbean.

Our mission is to make Puerto Rico the main telecommunications hub of the Caribbean Region by providing the most reliable telecommunications infrastructure and technology services.

-------------------

Althea: Frequently Asked Questions
The FAQ for you

We are excited to begin preparations for the Althea blockchain launch! The Althea blockchain is the future of telecommunications and consumer financial services. Creating a decentralized, secure, and reliable platform for Althea’s pay-per-forward bandwidth market.‌

What is Althea?
Why does Althea have a blockchain?
Do I need to know about or own crypto-currency to use Althea?
How does Althea currently operate?
How will Althea operate in the future?
How does the end user experience Althea Internet?
Why is Althea better than x, y and z?
How can I participate in the Althea Network?
Who is Hawk Networks?
How can I get involved in the Althea community?

What is Althea?

#whatisalthea

Althea is an Internet Service Provider (ISP) platform and blockchain that enables the coordination of multi-stakeholder networks. Althea decouples the service and infrastructure layers of Internet delivery, and coordinates transparent and programmatic revenue sharing.

Using Althea’s platform, networks can be built faster and more economically than ever before.

Althea’s flexible payment structure opens new markets and supports new entrants into the market, by enabling them to capitalize on underutilized bandwidth and real estate.

Why does Althea have a blockchain?

#whyblockchain

Last mile telecom infrastructure is siloed behind restrictive contracts and agreements that preclude an open bandwidth market. However, at deeper layers of the internet, bandwidth is sold between carriers as a metered commodity. Althea creates an open marketplace: from the data center down to the last mile of internet access.  Althea D-LTE (distributed LTE) will allow seamless carrier roaming, without cumbersome roaming agreements, directly to the device in your pocket.

This open telecom marketplace allows for both new entrants and existing carriers to interoperate and earn revenue. Althea’s multi-stakeholder model solves the friction of accessing revenue from rural, low income, emerging markets, and the complexity of capex and coordination of infrastructure build outs, including 5G and fiber.

The Althea blockchain is secured by a distributed network of token holders and validators that process transactions. By contrast, centralized payment solutions require permissioned coordination, and are often restricted by bespoke regulation or restrictions.

Similar to how the internet has allowed for global, open e-commerce, industry and trade—this borderless, robust and resilient payment layer will open new markets and increase revenue for a diverse group of participants.

Althea’s telecom blockchain will also open DeFi functionality to telecom carriers and users with Gravity, the Ethereum to Cosmos Bridge, where they can seamlessly leverage the liquidity of their networks for additional investment strategies. Because Althea is built using the Cosmos SDK and with Gravity functionality, carriers can also interoperate easily with other Cosmos blockchains using IBC (inter blockchain communication).

Do I need to know about or own cryptocurrency to use Althea?

Althea is easy for anyone to use, even if they don’t know about cryptocurrency.  Althea features an intuitive payment portal to add or remove funds, and payments within the network use stable coins, so the value doesn’t fluctuate.  A text message alerts users when their balance is low and users can add more funds using the onboard payment system and a debit card.

Althea participants who earn revenue for forwarding bandwidth or maintaining the network automatically receive payments to their router wallet.

How does Althea currently operate?

Althea’s novel pay-per-forward model operates on the xDai blockchain, representing 70% of xDai’s microtransaction volume over the last year (as of January, 2021). Althea routers seamlessly bridge payments in Ethereum and Dai to the xDai platform.

How will Althea operate in the future?

A typical Althea user will use a payment portal to add funds to their account. These will be transparently converted to Dai and, using Althea’s cross-chain Gravity Bridge, will be sent to the Althea blockchain. The Althea blockchain will be secured by a distributed network of token holders and validators that process transactions.

How does the end user experience Althea Internet?

Quickly, easily and seamlessly. Althea’s network delivers high-speed broadband to the end user, and  Althea’s upcoming distributed LTE service brings seamless carrier roaming to your mobile phone and smart devices, without cumbersome roaming agreements.

How does Althea improve on services provided by traditional ISPs?

• Structured as a smart contract on a blockchain, Althea’s innovative model for internet delivery decouples the service layer of the network from the underlying infrastructure, enabling investment into fiber networks in unique and sustainable ways.
• Althea’s pay-per-forward protocol allows a radical new approach to coordinating payments with bandwidth and network routing. This method is radical in its simplicity and efficient enough for near limitless scalability.
• Althea's pre-paid model and always-on Free Tier enable flexible payments for any budget and enable broadband access for more people.
• Hardware agnostic, Althea's platform can be utilized with fiber, LTE, and fixed wireless, enabling networks to grow and carriers to capture markets that were difficult to access.
  

How can I participate in the Althea Network?

There are four roles in an Althea network:

1. Bandwidth provider: Bandwidth providers play a passive role by running an Althea Node. They purchase a wholesale internet connection or serve as a middle mile provider and connect to an Althea Network.
2. Relay: Providing real estate and an Althea node which forwards bandwidth.
3. Network operator: A network operator gets paid for supporting and installing users (on a daily "uptime" basis) from live routers in the network. This role is supported by the Network Operations Center. (OSHA approved picture of installer)
4. Network Operations Center: A Network Operations Center (NOC) is run by companies who support the operators with technical support, logistics and planning. Currently Althea’s parent corporation Hawk Networks is the primary NOC, but we anticipate an ecosystem to grow.**

Who is Hawk Networks?

Hawk Networks Inc., is a Delaware corporation that has built the open source core, blockchain bridges and supporting software for Althea. Hawk Networks ensures that its development is supported and that end users can reliably use the system.

Hawk Networks also operates a Network Operations Center, NOC, that supports the build of new internet networks and forms strategically aligned partnerships.

While Hawk Networks is the first NOC, other NOCs will form to support additional networks, and large carriers will utilize the underlying protocol and blockchain as well.

How can I get involved in the Althea community?

If you’re feeling social and would like to come chat with us and the rest of the Althea community, you can visit our  or our Telegram (need invite link) or Discord channels.

If you'd like to read more, you might enjoy a visit to our blog. Feeling social? Find us on Facebook or Instagram. A little more visual? Visit our YouTube channel.

Link reference

Blog: blog.althea.net
Twitter: twitter.com/AltheaNetwork
Medium: medium.com/althea-mesh
FB: facebook.com/altheanetwork
Instagram:   instagram.com/altheanet/
discord: discord.gg/hHx7HxcycF
Reddit: reddit.com/r/altheamesh/
Youtube: youtube.com/c/AltheaNet

Announcing the upcoming Althea blockchain launch!

We are excited to begin preparations for the Althea blockchain launch! The Althea blockchain is the future of telecommunications and consumer financial services. Creating a decentralized, secure, and reliable platform for Althea’s pay-per-forward bandwidth market.

About the Althea Blockchain:

Last mile telecom infrastructure is siloed behind restrictive contracts and agreements that preclude an open bandwidth market. However, at deeper layers of the internet, bandwidth is sold between carriers as a metered commodity. Althea creates an open marketplace: from the data center down to the last mile of internet access.  Althea D-LTE (distributed LTE) will allow seamless carrier roaming, without cumbersome roaming agreements, directly to the device in your pocket.

This open telecom marketplace allows for both new entrants and existing carriers to interoperate and earn revenue. Althea’s multi-stakeholder model solves the friction of accessing revenue from rural, low income, emerging markets, and the complexity of capex and coordination of infrastructure build outs, including 5G and fiber.

The Althea blockchain is secured by a distributed network of token holders and validators that process transactions. By contrast, centralized payment solutions require permissioned coordination, and are often restricted by bespoke regulation or restrictions.

Similar to how the internet has allowed for global, open e-commerce, industry and trade—this borderless, robust and resilient payment layer will open new markets and increase revenue for a diverse group of participants.

Althea’s telecom blockchain further opens DeFi functionality to telecom carriers with Gravity, the Ethereum to Cosmos Bridge, where they can seamlessly leverage the liquidity of their networks for additional investment strategies. Because Althea is built using the Cosmos SDK and with Gravity functionality, carriers can also interoperate easily with other Cosmos blockchains using IBC (inter blockchain communication).

Interoperable blockchain built for interoperable telecommunications.

To claim your tokens from the airdrop:

1. Visit https://keys.althea.net/ and follow the instructions to create your wallet address
2. Copy your wallet address
3. Log in to your Republic account and visit https://republic.co/portfolio/althea
4. In the field provided, paste your wallet address and submit! [Note: Republic will never ask for your seed phrase. You should store this in a secure, private location and never share it with a third party.]
5. Expect an update on your distribution in the coming weeks!
6. To modify your address, visit https://republic.co/portfolio/althea and click the pencil icon next to 'Wallet'
   

Tokens will be deposited to your public address upon launch of the Althea blockchain.

Questions? Navigate to the FAQ below to learn more.

We look forward to launching the blockchain with the Republic airdrop participants and together changing the future of blockchain and telecommunications!

FAQ

What is the Althea blockchain?

The Althea blockchain is the technology powering Althea’s holistic, multi-stakeholder ecosystem. Thanks to this network, Althea can successfully align incentives as a decentralized ISP that empowers communities to build sustainable networks — both quicker and more affordably than any traditional internet service provider. Learn more at www.althea.net

What is a seed phrase?

A seed phrase is an input given to derive a private key. This is often a string of randomly generated words, which you need to use in order to access your tokens at your address.

Why do I need an address?

An address is sometimes referred to as the "public key." This is like an email address, or a digital wallet, and you need it because this is what enables you to claim your tokens as a participant of the Althea airdrop.

What should I do once I’ve generated an address?

Once you’ve generated an address, we recommend that you physically write down both your address and seed phrase on a piece of paper. After you’ve written them both down, you will want to input just your address into the Republic page where you will claim your tokens. Important: never share your seed phrase or private key! In the event you lose your seed phrase or private key, your public key/digital wallet may become irrevocably locked, resulting in a  total loss of its contents.

Where should I store my address and seed phrase?

We encourage you to write these down and put them in a safe place. Copy your address, being sure to “select all” and save to a secure location on your computer.

The safest and most secure place to store your sensitive seed phrase is offline, off of your computer, and on something physical, like a piece of paper.

Neither Althea or Republic have access to your address and seed phrase and will not be able to recover them.

What happens when I refresh the address generator?

Please note that when you refresh the address generator page, you will generate a new public-private key pair. This means that if you leave or refresh the page you will get a new address, and will lose all information from the previously generated address, including the seed phrase. This is because the page generates keys locally on your computer, and does not store anything itself.

When do I need to generate an address by?

You can submit and/or update your address via the Republic page up to the Last Call date. You will be notified of when the Last Call date is set, and reminded beforehand that the Last Call to submit an address is approaching to claim your tokens.

Why can’t I check my token balance yet?

Once the Althea blockchain launches, you will be able to check your token balance. Until the network is live, it’s not yet possible to check any token balances on-chain.

When will the Althea blockchain be ready?

We’re excited for the Althea blockchain to launch soon! Once the network is live, your token balance will be viewable and your tokens will be locked pending a network-wide governance proposal to activate them. You will need to use a Cosmos capable wallet, like Cosmostation or Keplr, to view your token balance.

How do I reach out with further questions?

Hello@althea.net

Discord::      discord.gg/hHx7HxcycF

How do I stay on top of Althea updates?

Blog:           blog.althea.net

Twitter:     twitter.com/AltheaNetwork

Discord::      discord.gg/hHx7HxcycF

Disclaimer and Limitation of Liability
The tokens granted to you through the Althea airdrop are provided ”as is ” and without warranty of any kind, either express or implied. To the fullest extent permitted by law, Althea discards all warranties, express or implied, regarding your use of the tokens. Althea does not warrant that the tokens accessible via the Althea airdrop are free of viruses or any other harmful components. You agree that your use of the wallet and tokens will be at your sole risk. Althea shall have no liability or responsibility for any: (i) user error such as forgotten passwords/seed phrase, defective transactions, or mistyped addresses; (ii) any unauthorized access to an address; (iii) server failure or data loss; (iv) any interruption, viruses, phishing, scamming, brute forcing, or other attacks against the tokens or a wallet. Please be aware that certain jurisdiction legislation does not allow the disclaimer of implied warranties, so the foregoing disclaimers may not apply to you.

To the fullest extent permitted by applicable law: (i) in no event will Althea be liable for any indirect, special, incidental, consequential, or exemplary damages of any kind (including, but not limited to, where related to loss of revenue, income or profits, loss of use or data, or damages for business interruption) arising out of or in any way related to the sale of the tokens or otherwise related to these terms, regardless of the form of action, whether based in contract, tort (including, but not limited to, simple negligence, whether active, passive or imputed), or any other legal or equitable theory (even if the party has been advised of the possibility of such damages and regardless of whether such damages were foreseeable); and (ii) in no event will Althea’s aggregate liability, whether in contract, warranty, tort (including negligence, whether active, passive or imputed), or other theory, arising out of or relating to these terms exceed the amount paid you for the tokens. The limitations set forth this paragraph will not limit or exclude liability for fraud or intentional misconduct of Althea. Some jurisdictions do not allow the limitation or exclusion of liability for incidental or consequential damages. Accordingly, some of the limitations of this paragraph may not apply to you.

Infrastructure is a shared dream for a better tomorrow, built today. It requires long term thinking, solid engineering, and most of all a shared dream for a better future.

Althea created Cosmos Gravity Bridge as infrastructure for the Cosmos community.

We are proud to announce Gravity Bridge - an independent and credibly neutral Cosmos-SDK blockchain providing the entire SDK community with a neutral bridge to Ethereum and eventually all major EVM chains.

Learn more about the exciting future at:
https://www.gravitybridge.net/post/the-gravity-bridge-blockchain

FOR IMMEDIATE RELEASE
Contact: LaShawn Williamson Wave 7 Communications, LLC
Phone: 877-356-0741 ext. 1012 3125 Kathleen Ave, Suite 229
Email: lashawn@wave7comm.net Greensboro, NC 27408

Wave 7 Communications, LLC is an Internet Service Provider, empowering the rural community of Enfield, NC to not only become users of their high-speed internet, but also serve as a key part of how the network expands and grows in their town through a $154,000 grant awarded to Wave 7 from the Internet Society and Truist. The Expanding Potential in Communities (EPIC) Grant program supports broadband initiatives in the southeastern United States, funded by Truist Financial Corporation’s Truist Cares* initiative, a $50 million philanthropic pledge to help rebuild communities in the company’s markets affected by COVID-19.

Wave 7 Communications was founded in February of 2020. The network was launched in June of 2020 as an answer to the lack of internet access in the rural town of Enfield. Wave 7’s innovative approach partners closely with organizations, businesses, residents and municipalities to locate infrastructure to deploy internet coverage.

“Residents living in rural areas often have limited choices for internet service, which results in slow speeds and a lack of flexible options. Our company is excited to serve the town of Enfield and surrounding areas with faster and more affordable internet. We understand the importance of broadband to the economy, e-Learning, and telehealth. Our goal is to give the town a community-based option for internet service,” said LaShawn Williamson, CEO of Wave 7 Communications.

A local non-profit organization, Community of Hope Center (COHC) has partnered with Wave 7 on this initiative by providing digital literacy classes. COHC will also assist with identifying 25 families who will be the recipients of one year of free-internet service from Wave 7 and receive one Chromebook per household as part of the grant funding.

As part of Wave 7’s further expansion in the town of Enfield, users will receive the internet via a small antenna on their roof and a smart router using the Althea system, which selects the most affordable, best connection available automatically. Members can control the cost of their service with no contracts. The average speed is 50-70Mbps, with many people receiving 100Mbps, and costs around $50 a month. The Wave 7 network ensures that none of their customers are without internet by providing an “always-on free tier” that delivers access to the basics of the internet at no charge.

For more information or to request a site survey at your location for internet service visit wave7comm.net.

About Wave 7 Communications
Wave 7 Communications, LLC (Wave 7) is a minority woman owned, internet service provider founded in February 2020 by LaShawn Williamson. Wave 7 is powered by Althea and provides an innovative mesh network solution to bring low cost, high-speed broadband access. Our mission is to deliver internet services in a sustainable way while greatly improving the overall connectivity of high-speed, secure, and affordable internet service. The goal is not to compete with the incumbent ISPs; however, Wave 7 is committed to filling in the broadband access gap that exists in communities by providing them with an affordable alternative for internet services.

About Althea (www.althea.net)
Althea is a new way to access broadband internet that enables growth in the network through a unique system of revenue sharing. Althea was founded in 2017 with the vision to bring the open market and interoperable peering from the data center to the field and empowering communities to build multi-stakeholder networks faster and more affordably than legacy telecom models.

Residents are encouraged to visit www.althea.net/waveseven and pre-register to show their interest.

The short answer:

• Smart contracts on blockchain make revenue share contracts possible between stakeholders of a broadband network.
• Revenue share contracts reduce the amount of risk capital required to build and operate a broadband network.
• Reducing dependence on risk capital makes it feasible for stakeholders to absorb demand risk.
• Demand risk reduction makes it possible to deploy broadband networks in underserved areas.

The longer answer: Broadband is as essential as electricity or water but is not as widespread. For context, 100% of homes in the U.S. have electricity and water but tens of millions do not have access to affordable broadband even though demand for broadband clearly exists. The unmet demand for broadband was clearly demonstrated during the COVID-19 induced lockdowns when stories of students doing homework in parking lots outside McDonalds started emerging.

So what is the fundamental difference between incentives of traditional infrastructure like electricity versus broadband? It boils down to demand risk.

Broadband investors assume demand risk when building infrastructure. Electricity or water utility investors assume no demand risk when building infrastructure.

Economic incentives of broadband infrastructure versus traditional infrastructure

To understand why unmet demand for broadband exists, let’s compare the economic incentives of broadband infrastructure with traditional infrastructure.

1. Revenue generation: Traditional infrastructure businesses like electric utilities generate revenue by earning a fixed rate of return on assets deployed in a regulated market and not by selling units of electricity. Broadband businesses generate revenue by selling units of data or leasing their infrastructure to other providers in a competitive market.

2. Profit distribution: Broadband investors generate 5–10x more profits in dense urban areas versus rural areas. Traditional infrastructure investors are mostly agnostic to population density because they earn a fixed rate of return based on assets deployed. Assets on which traditional infrastructure investors can earn a return is known as Rate Base or Regulated Asset Base.

3. Performance benchmarks: Traditional infrastructure investors will keep earning a return so long as their assets remain available for use subject to pre-agreed performance benchmarks like SAIDI (System Average Interruption Duration Index). Broadband investors have no such performance benchmarks because, in theory, broadband businesses operate in a competitive market.

4. Demand and competition risk: Broadband investors assume 100% of demand and competition risk whereas traditional infrastructure investors assume negligible competition and demand risk after infrastructure is deployed.

As a result, broadband infrastructure investors cannot obtain long-term debt financing the way traditional infrastructure investors can.

5. Investor objective: Broadband investors aim to deploy the least amount of resources to generate the maximum profit for shareholders. Traditional infrastructure investors aim to deploy the maximum amount of capital so long as Rate Base grows.

6. Customer pricing: Traditional infrastructure offers cost-based pricing to its customers because traditional infrastructure investors assume limited demand risk and therefore agree to a regulated rate of return. Cost-based pricing, as the name suggests, is price determined by what service delivery actually costs. Under cost-based pricing, the price paid per user should decline as the number of users increases.

In contrast, broadband infrastructure investors offer value-based pricing to their customers. Value based pricing is the price determined by a customer’s ability to pay for a service. Under value-based pricing there is no incentive to reduce prices other than competition risk. In the charts below, the x-axis represents years of operation and corresponding increase in customers.

Value-based pricing can work out well for customers in a competitive market. Unfortunately, in many countries, broadband competition risk is mostly theoretical. For example, in the United States, there are over 70 million people who have only one Internet Service Provider (“ISP”) serving them. When ISPs operate as a duopoly there is limited incentive to deliver better service to customers.

What does a smart contract for broadband infrastructure look like?

A smart contract is a computer program or a transaction protocol that needs no external action to execute the terms of a contract. A smart contract for broadband infrastructure uses network traffic data to make programmatic payments to network stakeholders.

An example of a smart contract for telecom and broadband has been built by Althea Networks. To date, Althea has used this system to build 14 broadband networks in underserved areas including one network in Abuja, Nigeria.

To get Internet service, an Althea user (“User Node”) loads money in their Althea router through a credit/debit card. The Althea router converts this payment to stable digital currency (“stablecoin”) that is pegged to the US dollar.

This stablecoin follows a price-aware protocol to make payments to Network Stakeholders. Network Stakeholders include neighboring customers (“Relay Node”), middle-mile fiber operators, building owners, cell tower owners, network technicians, capital providers, equipment suppliers, and spectrum owners. See the Althea Whitepaper for more details.

Payments made through the Althea system are programmable, visible to all stakeholders, and auditable on a public blockchain. This level of transparency makes it possible for any Network Stakeholder to join a revenue share smart contract in a trust-minimized way.

The Althea blockchain is secured by validators holding the Althea governance token (“ALTG”). ALTG token holders earn 10% of transaction fees flowing through a network as compensation for maintaining trust between Network Stakeholders.

Without Althea, a revenue share contract expects a Network Stakeholder to trust an Internet Service Provider (“ISP”) in two ways:

1. Honesty with revenue figures
2. Compliance with timely payments

Establishing and maintaining this level of trust is costly and time consuming which is why most ISPs end up signing fixed payment contracts with Network Stakeholders. Per such agreements, an ISP assumes 100% of demand and competition risk which in turn increases the amount of risk capital required to build a network. If this ISP makes a mistake in forecasting demand it will incur financial losses. Fear of losses causes ISP investors to become overly conservative and ignore rural and underserved areas.

Broadband is a necessity and every human is hungry for it. However, broadband is hungry for infrastructure. That infrastructure is hungry for capital. Unlocking capital requires predictable cash flows, bankable contracts, aligned incentives, and most importantly, trust. Smart contracts on a blockchain make it feasible to maintain trust between stakeholders.

How to build a broadband network with Althea

Step 1: Find a cell tower operator or middle-mile fiber operator (“InfraCo”) willing to enter in a revenue-share smart contract. OPTK Networks in Nebraska is one such InfraCo. OPTK’s fiber network spans over 10,000 miles and is present at several thousand points of presence (“PoPs”). These PoPs can be commercial buildings, cell towers, water towers or similar vertical assets.

Step 2: Obtain a list of PoP owners and inquire which of those owners are willing to enter into a revenue share contract.

Step 3: Invite private investors or equipment suppliers to fund/contribute Gateway Nodes and User Nodes against a revenue share. A single Gateway Node costs $3,000 and a User Node costs $500 in the U.S. Show private investors/equipment suppliers simple unit economics of a single Gateway Node at 100 users.

Step 4: Construct Gateway Nodes at PoPs where the building/tower owner is willing to enter into a revenue-share contract.

Step 5: Start marketing to users living within 1–2 miles of a Gateway Node. Encourage users to self-finance User Nodes and achieve bill reductions.

Step 6: When a user signs up, send a technician to their home, install equipment and activate service.

Important to note that, other than marketing expense which also gets capitalized, there are zero operating costs in the above structure.

A user’s monthly spend on Internet service will depend on data usage. For the average user, this will be $55/month if the user chooses to not pay for the home equipment themselves (see Metered Bandwidth by Justin Kilpatrick for more). A single Gateway Node can support at least 100 users. Unit economics of a single Gateway Node are as follows.

To incentivize local ownership of broadband infrastructure, offer Internet users bill reductions if they self-finance a User Node. A user’s bill reduction is determined by the Network Users and Total Capital Deployed. For example, if there are 100 Network Users and 1 Gateway Node, Total Capital Deployed is $53,000. In this example, bill reduction calculation will be as follows:

• User capex as / Total Capital Deployed = $500 / $53,000 = 0.94%
• Monthly fee for Capital Providers at 100 users = $1,500
• Monthly bill reduction = 0.94% X $1,500 = $14.1/month
• Net monthly bill = $41/month

Estimating global potential

Globally there are 4.96 million cell towers that provide mobile Internet coverage to the entire world, yet there are over 3 billion people who cannot access the Internet. Majority of these cell towers have excess capacity available to host Gateway Nodes at zero marginal cost through a revenue share smart contract.

At present, over 70% of these cell towers are owned by tower sharing companies or InfraCos while 30% are owned by telecom operators. InfraCos are incentivized to maximize tower utilization and tower cash flows but struggle to find creditworthy ISP customers as tower tenants in underserved locations. InfraCos also have full time network technicians on payroll who can do installations and maintenance.

If an InfraCo (eg Crown Castle, American Tower, Cellnex et al), a wireless equipment manufacturer (eg Ericsson, Ubiquity, Siklu, Parallel Wireless et al) and a marketing agency entered into a revenue share smart contract, they could build a broadband network without any upfront risk capital. Such networks will be able to operate profitably at much lower prices than a traditional ISP because they will not have to account for equity investor dividends, debt repayments or management salaries. Such networks can make it feasible to serve billions of people who live within a mobile Internet coverage area but are still not online.

For a further deep dive - check out the latest Althea community call with Asfi, Deborah and Justin - https://youtu.be/-Q-3-vXtyUE.

To finance this network, we will use Althea’s system to execute a revenue share smart contract between OPTK, cell tower operators, building owners, investors and network technicians (“Network Stakeholders”).

Althea’s system offers real-time payments and complete fund flow transparency to Network Stakeholders. This visibility makes a revenue share contract possible in a trust-minimized way. A revenue share contract for broadband matters because it reduces the amount of demand-risk assumed by broadband investors. We believe reducing and eventually eliminating demand risk assumed by broadband investors is required to bridge the digital divide.

“Broadband is an essential utility that should be available for everyone. Legacy models have left rural people behind, but Althea’s multi-stakeholder system empowers communities to build fast and affordable networks in sustainable ways. We are excited to work together with NetEquity and OPTK to bring better broadband to Nebraska, “ said Deborah Simpier, CEO and co-founder of Althea.

In the summer of 2021, Althea and NetEquity will select several areas in Nebraska to build out high speed fixed wireless and LTE networks that will use OPTK’s fiber network for backhaul. Users will receive the Internet via a small antenna on their roof and a smart router, which selects the cheapest, best connection available and pays the bill - all automatically. Customers can expect a download speed of 70-100 Mbps at a cost of $55/month.

“We have developed a deep appreciation of the culture and values found within the Althea community. This community is an inclusive meritocracy, it embraces diversity and is quickly becoming a talent magnet for telecom, broadband and finance professionals. This model can be operationalized and scaled globally to address the needs for fast, affordable broadband for everyone“ said Isfandiyar Shaheen, Founder and CEO of Net Equity.

About Althea:

Althea's unique cooperative vision for the internet brings peering from the data center to the field. Althea has been empowering communities to build multi-stakeholder networks faster and more affordably than legacy telecom models since 2018.

To date, the Althea platform has helped operationalize 14 networks in the United States and Nigeria.

www.althea.net @AltheaNetwork

About Net Equity:

NetEquity Networks is an infrastructure sharing start-up based in San Francisco, CA. NetEquity was incorporated in 2018 while its founder was an Entrepreneur in Residence at Facebook Connectivity. NetEquity’s mission is to make high speed Internet access affordable for all within a decade.

www.netequity.net @netequity1

The Althea team along with other core Cosmos contributors chose the name 'Gravity', the logo, and imagery very intentionally for Gravity Bridge to represent the force drawing utility toward the Cosmos Hub. Gravity DEX, and all other future products coming together under the Gravity umbrella are destined to expand and enrich the utility of the Cosmos Hub and the wider ecosystem, strengthening the pull of Gravity even further.

The goal is as straightforward as it is ambitious. To build a force as inevitable as gravity. A force that pulls capital to the Cosmos Hub and IBC-enabled chains. You don't do this by forcing people to move their capital. Migrating years of infrastructure and onboarding from Ethereum is a hard climb.

Using Gravity, well, it's as easy as falling.

Execution context sharing

Gravity Bridge unlocks enormous amounts of currently inaccessible capital on the Ethereum chain by providing a shared execution context on Ethereum, governed by the Cosmos Hub.

21000 gas is charged for any transaction as a “base fee”. This covers the cost of an elliptic curve operation to recover the sender address from the signature as well as the disk and bandwidth space of storing the transaction. -- Ethereum Design Rationale

By using the external consensus of the Cosmos Hub blockchain to package a single Ethereum transaction out of many different users' operations, every operation becomes 21,000 gas cheaper than if it had been run individually.

We've already seen how Gravity Bridge can perform cheaper ERC20 sends. But Gravity Bridge can also execute batches of arbitrary contract calls. The value of arbitrage opportunities on Ethereum is driven by gas prices. All trades on Ethereum are arbitraged up to the point where the gas cost exceeds the value of the rewards.

Lower gas prices create hundreds of millions of dollars of arbitrage opportunities. This creates more traffic, which raises gas prices, which washes away potential arbitrage profits, eventually resulting in an equilibrium.

So what happens when we slash that cost across the board? Millions of dollars of trade opportunities that were previously unreachable become accessible.

Using the aggregate efficiency of the Gravity Bridge's shared execution context, provided by the Cosmos Hub validator set, this efficiency is possible and accessible to everyone. This creates a favorable situation  for participants on either side of the bridge as the extra efficiency allows DeFi to do more.

Minted rewards

Gravity Bridge brings unprecedented power to the stakeholders of a Cosmos chain by giving them the ability to mint tokens as a relayer reward, effectively providing a gas subsidy fueled by inflation of the Cosmos chain's native token.

On its own, using inflation to drive Ethereum traders to the Cosmos side of the bridge is simply an unsustainable growth hacking strategy. But when combined with context sharing it becomes a feedback loop.

It's as easy as falling

A gravity well is a feedback loop. A star brings in mass, which increases its gravitational pull, which allows it to bring in more mass. The Cosmos Hub has the 'mass' required to generate an enormous feedback loop. Gas subsidies drive more users to access Ethereum DeFi via the Cosmos Hub and Gravity Bridge.

More bridge users drive lower gas costs through execution context sharing. Lower gas costs open up millions of dollars of Ethereum arbitrage opportunities that are only accessible via the Cosmos Hub.

As this loop repeats, the gravity well grows stronger, drawing more users and capital to the Cosmos Hub. Gravity Dex gains volume, becoming a vast marketplace that grows in value as more assets are moved on to it, driving all Cosmos native DeFi, and further enhancing the capital advantages of being on the Hub.

"It's one of the strongest ways to add more utility to the Cosmos Hub," enthuses Tendermint CEO, Peng Zhong. "The interchain Gravity DEX connected to Ethereum via Gravity Bridge will attract new chains to start trading and creating liquidity, traders to start arbitraging, and developers, who now have a new path to start listing their tokens permissionlessly and getting people to invest."

All of this increases the pull of the Cosmos Hub and takes us back to the first step of this virtuous cycle.

Timelines

Protocol Audit: We are working with Infomal Systems to have a protocol audit complete in early to mid June.

Hub signaling proposal: Submit signaling proposal for Gravity on the Cosmos hub with the audit in early to mid June

Gravity bridge/Althea testnet: The unincentivized testnet will be online with our new changes this week, as we prep for the incentivized testnet.

Althea has been hard at work making this vision a reality and bringing Gravity to the Cosmos Hub and our own chain.

We believe in building cooperative systems that bring people together for the benefit of every user and we extend that principle beyond the internet and into financial services.

Our dream is to use the Gravity Well to drive enormous capital efficiency for our Althea internet users. Lowering the cost of onboarding and increasing Althea users access to financial services.

Financial empowerment and the ownership and choice of communications are the key pillars to solving the digital divide. Recent numbers from Microsoft indicate that over forty percent of people in the US do not access the internet at broadband speeds, a problem that is often relegated to the lack of infrastructure, making the case that there simply isn’t enough “internet pipes” to reach everyone. A quick glance at the amount of under-served in major urban areas, where there exists a diversity of infrastructure shows the lack of nuance to this hypothesis. Portland, Oregon, for example, recently published a feasibility study that showed that 13% lacked access to broadband and New York City’s request for proposal indicates that 15% of the population have no access to internet at all. This is despite billions of dollars of federal grants  and the capital investment of many major telcos.

I have had the unique opportunity over these past few years to interact with both under-served rural and urban communities and have gained a much broader understanding of the extremely nuanced and interconnected problems that plague many Americans and their lack of access to broadband. The key surprise and takeaway was that it is deeply rooted in the lack of access to financial services.

For many people, we do all of our business online and use banks with ease. We can choose where to put our money, how we want to pay for things, and earn interest or invest. These financial services are empowering things - we can leverage what we have to work for us. But, for many of our neighbors (the FDIC in 2019 reports that 22% or over 60 million people in the US are underbanked or unbanked), they bank in very different ways. They use the grocery, convenience store or Walmart to cash their paychecks, pay expensive fees for pre-paid debit cards or money orders, send money to family and friends with expensive services like Western Union, and never have the opportunity to have ownership, choice, and empowerment of their money.

With only cash in hand, this can lead to difficulty in paying for an internet bill. Exchanging cash for money orders or a prepaid debit card is expensive and has to be done in person, making that monthly internet bill even more costly and difficult to stay timely with. This rigidity of a monthly recurring payment, sent often by mail or email, can also pose a hardship for people who may have intermittent income, or work so many hours that it is difficult to take the time to exchange cash. This can lead to falling behind in the monthly internet payment and eventually to  “broadband debt” as that bill becomes so large and goes to a collection agency. Those with large amounts of broadband debt may find that they are unable to set up any other internet services without first paying a large bill. This also preludes them from taking advantage of many subsidized programs like Comcast Essentials, which requires a monthly payment of $15 a month AND to be in good standing with Comcast.

We see this divide in cell phone access as well, but what has emerged are flexible payment structures, with cellular carriers seeing the opportunity for market share to the approximately 79 million prepaid subscribers in the US. In fact, “data credits” are bought and sold throughout the world in much the same way, which leads to several unique insights into the interconnection of accessibility of financial services and telecommunications.

In emerging markets with a larger underbanked population, we can look to the success of M-Pesa, a branchless mobile payment service offering an easy way to transfer money, and pay for goods and services (Lipa na M-Pesa). Further, M-Pesa users can also access the empowering functions of credit and savings, all with a mobile device. M-Pesa started in Kenya in 2007 and is now widespread, with presence in Tanzania, Mozambique, DRC, Lesotho, Ghana, Egypt, Afghanistan and South Africa. In Kenya, this has spurred adoption rates, with internet penetration growing substantially year-over-year. This example shows an interconnection of financial empowerment and telecommunications that can be translated into the US market as well.

The US is currently focused on enabling more broadband access, and, rightly so. Access to broadband improves outcomes in education, healthcare, and is proven to have numerous economic benefits. Broadband access is not just about building more infrastructure, however, we need to build systems that truly empower people to have ownership and choice.

Althea is excited to announce the completion and test of the first transaction of a Cosmos based token (ALTG) to test Ethereum (Goerli) using the second iteration of Althea Gravity, the Cosmos <> Ethereum bridge!  Earlier this year, Althea began testing the Gravity bridge sending Ethereum to Cosmos and back again. This additional functionality of the bridge enables Cosmos native assets and Atoms to be sent to Ethereum addresses and to DEXs like Uniswap, all with significant gas savings.

Althea’s core principles of open, interoperable telecom systems inform everything we do, including the work on the Gravity bridge. The interoperability between Cosmos chains with IBC and the Gravity bridge's interoperability between Ethereum and erc20 tokens opens new possibilities and market opportunities. In Althea, users currently load up their routers using a debit card and a simple Wyre plugin, seamlessly onboarding liquidity which then routes through a bridge to xdai, where Althea’s router-to-router payments account for over 70% of the volume of micro-transactions of the xdai network. Now, the Gravity bridge will onboard this liquidity through Ethereum to the Althea blockchain and the Cosmos ecosystem empowering the world's bandwidth users to not only have choice and ownership of their internet, but also leverage their liquidity to work for them.

Gravity’s batching enables extremely gas efficient transactions from Ethereum to Cosmos and back again, enabling a variety of exciting DeFi implementations, including the work of Sommelier and it’s “coprocessor to Ethereum”. Liquidity Providers (LPs) will be able to use the Sommelier to author and execute complex, and automated transactions, such as portfolio rebalancing, limit orders, batched orders, as well as a host of other features that traders have come to expect from CeFi, but that are not currently available in DeFi.

Justin Kilpatrick, Althea CTO and co-founder, remarks “Gravity is the first bridge efficient enough to *work with* Ethereum DeFi rather than just 'bring it somewhere else'. It's cheaper to do your trades through Gravity than it is to just use Ethereum.”

Because batched Gravity transactions share an execution context they can avoid the base fee that every Ethereum transaction pays to be processed by the network. For the vast majority of DeFi operations the base transaction fee (21,000 gas) makes up nearly half of all gas used.

With Gravity V2, native Cosmos assets can be bridged to Ethereum, which will make it easy for projects to launch a new blockchain with Starport and sell their tokens on Uniswap. Gravity v2 can also be used to compensate relayers using a chain's native token. No Cosmos chain can print more DAI, but it can obviously mint more of it’s own native token. Subsidizing gas with Gravity V2 could fundamentally change the financial dynamics of using a bridge and allow for fee-free transactions for users.

Gravity will also interact with upcoming Cosmos DEXs like B-Harvest’s Gravity DEX, coming as soon as Q2 this year!
The possibilities of Gravity and the multi-chain future are numerous.  The open and interoperable future of connectivity and finance is here and we are excited to be contributing to that ecosystem.

*****

We have just concluded a series of test nets, with the next upgrade to pre-production and iteration planned for later this week. Incentivized test nets to follow. We had a large number of participants in these early test nets which allowed us to experience different edge cases.

Check Twitter or Discord for updates.

Join us on Discord - https://discord.gg/jnYg8dd

Twitter @Altheanetwork

First, the high level overview: Gravity lets people transfer tokens from Ethereum to Cosmos and back again by locking up tokens on the Ethereum side, and minting equivalent tokens on the Cosmos side. Unlike many bridges, Gravity is completely non-custodial- you only need to trust in the security of the Cosmos chain, not some third party bridge administrators who could run off with your funds.

Here's how Gravity can be used:

• You send 25 DAI to the Gravity contract on Ethereum, specifying that your address on Cosmos should receive the DAI.
• Validators on the Cosmos chain see that this has happened and mint 25 DAI on Cosmos for your address.
• Now the DAI can be sent to other accounts on Cosmos, used in Cosmos applications, transferred to other Cosmos chains via IBC, etc.
• To go back to Ethereum, you send the DAI to Gravity module on the Cosmos chain, specifying that your address on Ethereum should receive it.
• The Cosmos validators burn the DAI on the Cosmos chain, and send an Ethereum transaction causing the Gravity.sol contract to send the 25 DAI to your Ethereum address.

Parts of Gravity

Gravity consists of 4 parts:

• An Ethereum contract called Gravity.sol
• The Gravity Cosmos module
• The orchestrator program which is run by Cosmos validators alongside the Cosmos blockchain's code
• A market of relayers, who compete to submit transactions to Ethereum on behalf of the Cosmos validators

Gravity.sol

Gravity.sol's main function is to lock up ERC20 tokens on Ethereum, so that they can be minted on Cosmos. The tokens locked in Gravity.sol back the tokens on Cosmos.

The current Cosmos validators control Gravity.sol, in proportion to their validation power on Cosmos. Gravity.sol stores a checkpoint- this corresponds to the active validator set on the Cosmos side at a given moment in time. A checkpoint is a set of addresses and their validation power. The more validation power an address has, the more its vote counts. The methods on Gravity.sol are public, and can be called by any relayer, but they must contain the signatures of 2/3s of the validator set in the current checkpoint.

updateValset

The validator set on Cosmos is always changing, as Cosmos token holders delegate tokens to different validators, changing their power. The Cosmos validators use a method called updateValset on Gravity.sol to update its stored checkpoint to the latest validator set. If a validator does not sign off on an update, or worse, signs a fraudulent update, they can be slashed, losing funds on Cosmos.

sendToCosmos

To transfer tokens from Ethereum to Cosmos, a user calls the sendToCosmos method. This method receives a number of ERC20 tokens from the user and locks them in Gravity.sol. It also emits an event which lets the Cosmos validators know that they should mint that number of tokens for the user on the Cosmos side.

submitBatch

Users send tokens on Cosmos to the Gravity module and specify which address on Ethereum should receive them. They also attach a fee to compensate whoever relays their transaction. Tokens are transferred from Cosmos to Ethereum in batches, to save on gas.  The Gravity module creates batches of transactions of one type of token, and then the validators sign them.

Relayers can submit batches to the submitBatch method. Gravity.sol:

• First checks that a batch has signatures from 2/3s of the validator set in the current checkpoint.
• Then checks that the batch has a nonce that is higher than any other batch of the same token type that was already submitted.
• Then sends ERC20 Ethereum transactions transferring the tokens to their destinations. Gravity.sol also sends the fees on the transactions to msg.sender, the relayer.

The Gravity Cosmos module and the orchestrator

The Gravity module is a module built on Cosmos SDK. Cosmos blockchains consist of many such modules, being run by in lockstep by their validators. Every validator runs the exact same deterministic logic as every other validator. The validators come to consensus on the state of a key-value store. Individual modules own different parts of this state.

The Gravity module is responsible for minting Cosmos tokens representing ERC20 tokens on Ethereum. It works closely together with the orchestrator. The orchestrator is a program that runs on every validator, alongside the Cosmos code. Since Cosmos modules on every validator are running the same logic in lockstep, they cannot do things like sign messages or transactions with a validator's unique key. Validators running a chain with the Gravity module installed use the orchestrator to do these things.

Ethereum oracle

The Gravity module must know what is going on on the Ethereum chain. This way it is able to mint tokens on Cosmos for people when they send tokens to the Gravity.sol contract on Ethereum. It also allows it to see when a batch of transactions going from Cosmos to Ethereum has been successfully processed.

Every validator on a Cosmos chain with the Gravity module must also run an Ethereum full node. The orchestrator on each validator monitors Ethereum events using this full node. There is a parameter called EthBlockDelay. Orchestrators monitor events this far back on the Ethereum chain. This delay is to allow the validators to have confidence that a given event is fully final on the Ethereum chain, and to make sure that every validator has the same view of the chain. When the orchestrator sees an event from the Gravity.sol contract, it puts the event into a Cosmos message, signs it, and broadcasts it to the Cosmos chain.

The Gravity module receives these messages and keeps track of which validators have voted for each individual Ethereum event. When an event crosses the threshold of 2/3s of the validation power having voted for it, it is considered "observed", and it changes the Cosmos state.

When a SendToCosmosEvent is observed, the Gravity module mints tokens for the destination address.

When a TransactionBatchExecutedEvent is observed, the Gravity module eliminates the batch that was executed from the batch pool, and cancels any earlier batches of that token type. More on this process later.

Transaction batching

Transactions going from Cosmos to Ethereum are put into batches before being sent to the Ethereum chain. This is because it is expensive to process transactions on Ethereum, and batching splits most of the cost up among all transactions in the batch.

When a user wants to send tokens from Cosmos to an address on Ethereum, they send the tokens to the Gravity module. Gravity deducts the tokens from their account, then puts a transaction in the transaction pool. The transaction pool stores all Cosmos to Ethereum transactions that are not yet in a batch.

Anyone can send a transaction requesting the Gravity module assemble a batch of a particular token type. This will most likely be done by relayers who wish to relay a batch of tokens for profit. When the Gravity module receives this message, it follows this procedure to assemble the batch:

• Find all transactions for the token
• Choose the 100 transactions with the highest fee and put them in a batch. The number of transactions may not be 1oo in the future. It may be configurable, or even dynamic.
• If there is a batch in the batch pool with a higher sum of fees than the newly assembled batch, discard the new batch. If not, include it in the batch pool. More on this later.

It then puts the batch into the batch pool. Once a batch is in the batch pool, all the validator's orchestrators make signatures over the batch and submit them in Cosmos messages. Each signature is stored alongside its batch.

Relayers are continually monitoring the batches in the batch pool. Once a batch has signatures from at least 2/3s of the validator set in the current checkpoint stored by Gravity.sol, it can be submitted to Gravity.sol. Relayers can choose for themselves whether submitting a given batch would be profitable, and submit the ones that are. If the fees included in the batch are higher than the cost of gas to submit it to Ethereum, it is a profitable batch. Of course, this is a very nuanced calculation which takes into account current average gas prices on the Ethereum chain, congestion on the chain, and the exchange rate between that token and Ethereum gas. This is why we have a free market of relayers.

Once a batch is executed on the Ethereum chain, Gravity.sol emits a TransactionBatchExecutedEvent as detailed above. After EthBlockDelay blocks on Ethereum, the validators submit the event to the Gravity module. When this event is fully observed (as detailed above), the batch is removed from the batch pool, since it is complete. Any batches of the same token with a lower batch nonce are also removed from the batch pool, but their transactions are also put back into the transaction pool. This is because we can be sure that it is impossible to submit them, since their batch nonces are too low. This mechanism allows transactions that were assembled into unprofitable batches to have a chance to make it into a profitable batch.

As mentioned above, the Gravity module will only assemble a new batch if there are no batches already in the batch pool with a higher sum of fees. This is to prevent the following scenario: If there were many transactions in the transaction pool with an unprofitably low fee, and a few coming in every block with a high fee, each high fee transaction might end up in a batch with a bunch of unprofitable transactions. These batches would not be profitable to submit, and so the profitable transactions would end up in unprofitable batches, and not be submitted.

By making it so that every new batch must be more profitable than any other batch that is waiting to be submitted, it gives the few profitable transactions that come in every block the chance to build up and form a batch profitable enough to submit.

Validator set updates

Validator set updates keep Gravity.sol's knowledge of the validator set on Cosmos current. This is important because Gravity's trust model requires that trusting Gravity.sol is the same as trusting the Cosmos chain running it. A validator set update must be signed by at least 2/3s of the current validator set in the checkpoint stored by Gravity.sol.

Anyone can send a Cosmos message requesting a validator set update. When this happens, the Gravity module saves a snapshot of the current validator set. The validators then sign this snapshot. Once it has the signatures of 2/3s of the validator set in Gravity.sol's current checkpoint, a relayer can submit it to the Ethereum chain. There are a few subtle points to note:

• The Gravity module does not actually know what the current checkpoint stored by Gravity.sol is. Because of the EthBlockDelay, all of the Gravity module's information about Ethereum is out of date. It is up to relayers to decide whether a validator set update has enough signatures to be submitted.
• There is no reward to relayers for submitting a validator set update. Unlike a transaction batch, there are no fees. This may be fixed in a later update. There are many ways it could be done, on the Cosmos side or the Ethereum side. But for now, the validators of a Cosmos chain running Gravity will likely just set up an altruistic relayer which is responsible for submitting validator set updates.
• Gravity.sol is not a Cosmos light client and it does not have knowledge of the full Cosmos consensus and staking process. It is basically a multisig whose signatories are intended to be the validator set of a particular Cosmos chain. We use some slashing on the Cosmos side to ensure that this is always the case.

Slashing

Slashing is an important part of Gravity's security. Validators who behave incorrectly on the Cosmos or the Ethereum side of the bridge are slashed on the Cosmos side. This is what links the security of the Gravity bridge to the security of the Cosmos chain it is run on.

Liveness slashing for batches and validator sets

By participating in the Cosmos consensus process, a validator is implicitly stating that all validator set updates and batches produced by that process are correct. If that validator does not sign these validator sets and batches, the validator is not operating correctly. After a grace period, it will be slashed for not signing.

Slashing for fake batches and validator sets

It is possible to submit batches and validator sets to the Gravity.sol contract which were never produced by the Cosmos chain, as long as they are signed by >2/3s of the validators in the contract's current checkpoint. To prevent malicious validators from forming a cartel to steal from the bridge with a fake transaction batch or validator set update, Gravity slashes validators whose signatures appear on batches and validator set updates that were never produced by the chain. This is accomplished through an evidence mechanism, where anyone can submit evidence of a signature over a fake batch or valset, and the chain will slash the validator responsible.

Join us on Discord! or Sign up Here for Validator Updates.

A big thank you to everyone who participated and took the time to setup a node. We're looking forward to a bright future for the Althea blockchain when it's launched next month.

Our goal is to keep this testnet running through multiple upgrades and code changes, all the way through to the Althea Gravity Incentivized Testnet, which will be our last testnet before the launch of the Althea chain. Anyone is welcome to join at any time, simply follow the instructions at github.com/althea-net/althea-chain.

Testing results: Day 0

Our first day of testing revealed several issues. Mainly around how validator set updates are generated and signed.

Validator set updates are messages that go from Cosmos -> Ethereum to synchronize the validator set in the Gravity Ethereum contract with the changing validator set.

The Gravity module on the Cosmos side of the bridge will automatically generate a new update every time the validator set power changes more than 1%.

In our small scale testing this worked perfectly. But as more than 25 validators joined the set validator set updates started being generated every block.

These extra validator sets are obviously not required, since actual voting power isn't changing. But due to rounding errors involving validator rewards the condition was always tripped.

Gravity orchestrators, run by the validators. Are responsible for signing these validator set updates and then relaying them to Ethereum. Due to a design oversight the orchestrator process could only sign updates at a maximum speed of one per block.

If validator sets are generated at a rate of once per block and can only be signed at a rate of once per block, falling behind is just a matter of time.

One of our testing goals for testnet1 is to check that Gravity slashing works. One of Gravity's slashing conditions is to slash a validator who has left a validator set update unsigned for more than 16 hours.

Since no one's orchestraor could keep up 16 hours after launch everyone was slashed and the chain halted.

This is obviously really valuable testing. Informing us of both implemetnation problems and a minor design issue.

• we have re-architected the validator set update signature system to handle up to 100 signatures per block. Allowing validators to easily catch up and keep up.
• we have tweaked the validator set generation condition not to produce spurious validator sets. This fix will need to be revisted in greater detail

Other fixes from day one

• Properly play back interleaved events in order: If a deposit and a withdraw occur in the same block they where not brought back to the Cosmos chain in the correct order by the oracle.
• Delegate Ethereum and Cosmos keys where not saved and loaded from the genesis file along with the rest of the Gravity state
• Handle a condtion where event nonces could get out of sync due to an improper state lookup
• Two or three obscure crashing condtions are now handled

What's next?

Now that have these issues fixed we can move on to setting up niceties like a block explorer and web wallets.

Exciting things are coming to the Althea testnet in the next few weeks. Including the first Gravity tokens sent over IBC and the first Cosmos based asset to ever set foot on Ethereum!

Learn more about Gravity here - https://github.com/althea-net/althea-chain

Chat - https://discord.gg/CjJwMJdb

We're proud to announce the Ethereum-Cosmos bridge, Gravity, which we will to bring to the Cosmos Hub with a governance proposal later this year. Gravity is an evolution of the Peggy bridge, which we have been working on together with the Interchain Foundation for much of the past year. Gravity is designed specifically for the Cosmos Hub. Gravity is intended to pull as many transactions and as much value as possible into the orbit of the Hub by bridging it directly to Ethereum.

Gravity is engineered to be exactly as secure as the Cosmos Hub. That is, if you trust the Hub to secure your bridged assets, you can trust Gravity to bridge them. We are working with Informal Systems to prove that this is the case. Since the Hub is backed by billions of dollars of staked Atoms, Gravity will be very secure. Once our governance proposal passes, and Gravity is installed on the Hub, Cosmos Hub validators will directly participate in the operation of the Gravity bridge. Since the Hub is backed by billions of dollars of stake, Gravity will be the most secure bridge to Ethereum in production.

Gravity is also engineered to be very efficient. The Ethereum contract is highly optimized to use as little gas as possible, and transactions are batched. This results in very cheap and frictionless transactions from Ethereum to Cosmos and back. We believe that this is important to pull as much value onto Cosmos as possible.

Gravity will be able to bring ERC20 assets from Ethereum into Cosmos, but it will also be able to bring Cosmos assets to Ethereum. $ATOM and every other asset in Cosmos ecosystem will trade on Uniswap and other Ethereum AMMs, bringing a huge amount of liquidity and value to these assets.

At Althea, efficient and secure Ethereum to Cosmos transactions are very important because it is how our users around the world pay for their internet bandwidth. We've worked very closely with the Interchain Foundation to get to this point. We've already run two testnets, and are looking forward to hosting an incentivized testnet in late Q1 2021. There will be prizes available for validators, relayers, and hackers.

Althea will be shepherding Gravity through the testing, audit, and implementation process on the Hub. We are committed to playing a long-term roll upgrading, documenting and supporting Gravity over the coming years.

If you're interested in participating in the testnet as a validator, relayer, or hacker, get in touch at gravity@althea.net, or view the source code at github.com/cosmos/gravity-bridge.

• Our original test network grew from a few dozen subscribers in one town, to hundreds of subscribers stretching across 30 linear miles and encompassing 5 towns.
• Other network operators built networks throughout the US, as we refined the process of planning, deploying and growing networks, and by year’s end we had live networks in six states and Nigeria.
• Althea’s unique ability to coordinate and incentivize users for hosting hardware enables the coordination and cost of numerous 5G small cells possible.
• Partnering with NetEquity.net we are developing a series of financial products, similar to the financing for solar build outs, that makes user owned Fiber to the Home not only a reality, but a scalable, repeatable process, which funnels an on-going source of capital into gigabit fiber networks.
• We learned that not only could we build a payment layer that seamlessly on-boarded average internet consumers and transmitted payments to upstream providers in a permissionless, peer-to-peer way, but also that this same system could provide financial freedom to a great number of under-banked people. All of the power of a distributed  bandwidth marketplace and home bank in one Althea box.
• Althea partnered with Iqlusion and others in the Cosmos ecosystem, to build the Peggy bridge and we closed 2020 with a fully functional test net and moving transactions from Ethereum to Cosmos and back again.
• In the first quarter of 2021, we look forward to the much anticipated launch of the Althea blockchain! The Althea blockchain enables a seamless funnel for capex, that aligns the incentives of users and shareholders.

2020 was quite a year, and one that accelerated the transformation to a digital environment. This change towards distance learning, telehealth and zoom based work also highlighted what many of us working in broadband already knew - The digital divide between those who have access to great internet and those who do not, is actually a gaping chasm. Those who have access to low cost, high quality internet and those who do not is not only about the strands of glass or other infrastructure - in fact 40% of the US doesn’t access the internet at broadband speeds.

The last mile infrastructure as we know it now is brittle and siloed, making each connection to your home or device a monopoly. Althea imagines a different and better world - the same open marketplace that telecom carriers enjoy at the core of the internet, the IX, brought all the way down to your home or even your mobile device. Your router selects and pays the best upstream provider on a second-by-second basis. This unique cooperative vision also empowers many new entrants to participate and earn revenue for hosting different parts of the infrastructure of the network, enabling a multi-stakeholder model for a network, instead of the centralized and monopolized structure we see today.

This multi-stakeholder model utilizes existing vertical assets and infrastructure, like homes, businesses, or farms, to build these decentralized networks which results in networks that can be built much faster and with less capex than the existing, legacy models. Throughout 2020 we iterated on this core concept with fixed wireless networks. Our original test network grew from a few dozen subscribers in one town, to hundreds of subscribers stretching across 30 linear miles and encompassing 5 towns.

Other network operators built networks throughout the US, as we refined the process of planning, deploying and growing networks, and by year’s end we had live networks in six states and Nigeria. Althea networks are interconnected and decentralized - providing failover across multiple backhaul connections and seamlessly switching without end user disruption.

On Christmas eve 2020 we went live with a network in the desert of SE Oregon, a very remote area with challenging hills and bluffs, proving that Althea can be built quickly and affordably just about anywhere. We have many more network operators looking to start networks in 2021 and are excited about the path to a fully operationalized network deployment process.

The real excitement is seeing the full stack of Althea networks coming to fruition. From fiber run along the middle mile and to the home, then an interconnected fixed wireless, and finally, distributed LTE to the device level. This past year we have laid the foundation, partnering with the University of Washington and the Tacoma Althea network, we began to deploy and test different distributed core infrastructure (open5gs.org) with the Althea stack, deploying user-owned LTE infrastructure to the home. The right-of-way complexity of deploying large scale 5G networks has been the singular issue to mass 5G roll out.  Althea’s unique ability to coordinate and incentivize users for hosting hardware enables the coordination and cost of numerous 5G small cells possible.

Another important upcoming milestone is user-owned, Althea fiber infrastructure. Partnering with NetEquity.net we are very excited to build some of the first user-owned incentivized fiber networks. In fixed wireless Althea networks, users buy $200-$400 worth of antennas and can use this hardware to earn revenue, reducing their cost for bandwidth or earning income. Fiber networks can be built in similar ways, although the initial cost is higher - $1500 - $3000. Partnering with NetEquity.net we are developing a series of financial products, similar to the financing for solar build outs, that makes user owned Fiber to the Home not only a reality, but a scalable, repeatable process, which funnels an on-going source of capital into gigabit fiber networks. These dense fiber networks move the middle mile from the legacy telcos to the home, pushing open access, Althea networks from the core to the end user.

The Althea decentralized and multi-stakeholder network revolutionizes how bandwidth is bought and sold. This open bandwidth marketplace spans hardware, technology, infrastructure and borders. The Althea blockchain is the only blockchain built for the needs of  this robust, extremely reliable global network.

Over the last year, we have rolled out network after network using existing blockchain technology, such as Ethereum and xDai, so that end users can use a debit card to easily buy Ethereum that bridges seamlessly to the stable xdai currency. From these experiences, we learned that not only could we build a payment layer that seamlessly on-boarded average internet consumers and transmitted payments to upstream providers in a permissionless, peer-to-peer way, but also that this same system could provide financial freedom to a great number of under-banked people.

The biggest surprise of maintaining and operating an internet network, was how many people were using the corner store for their daily banking activities, and that these same users were loading several hundred dollars into their routers. The Althea blockchain will enable these same users to do things like earn interest and take out a collateralized loan, empowering them with the financial freedom of banking all from their Althea router. All of the power of a distributed  bandwidth marketplace and home bank in one Althea box.

To build this interoperable platform, we needed to build the Althea blockchain on the  interoperable blockchain platform, Cosmos. For that, we needed a bridge from the ease of on-boarding and decentralized finance of Ethereum to a lower transaction cost and sovereign Althea blockchain, built on Cosmos. So, we set to work in earnest building the first decentralized Ethereum to Cosmos bridge, Peggy. Althea partnered with Iqlusion and others in the Cosmos ecosystem, to build the Peggy bridge and we closed 2020 with a fully functional test net and moving transactions from Ethereum to Cosmos and back again. From this collaboration, we formed a lasting partnership that will build, maintain, and develop bridges and decentralized finance on the Cosmos platform for years to come.

In the first quarter of 2021, we look forward to the much anticipated launch of the Althea blockchain! The Althea token distribution plan focuses on progressive decentralization, ensuring an always-on robustly engineered platform that then progressively decentralizes the ownership of the network. This fairly allocates tokens to the 8400 airdrop participants, early investors, and new token holders and enables an exit to the community over time.

This plan is the elegant solution to funding the capex of network build out and operation, without the untenable debt that legacy telcos carry. The reality is that the legacy telco models do not work for anyone - they take on debt to build large monopolized infrastructure systems with rent that is too high, creating broken systems for users and shareholders alike. The Althea blockchain enables a seamless funnel for capex, that aligns the incentives of users and shareholders.

We will also be hosting incentivized test nets, with a “honey pot” of Cosmos’ Atoms in the Peggy bridge for those who participate or who can capture the funds. After testing and auditing we will be navigating existing users to the Althea blockchain.

Althea, the platform of an open bandwidth marketplace, is ready - we have iterated on the build, operation, and market potential of decentralized networks in the field. In 2021 we will boldly launch the internet infrastructure and payment layer of the future, setting in motion something beyond our team. Myself, Justin, Jehan and the crew at Althea will continue to build tools and support the functionality of the network, but this distributed system will grow into a greater self-sustaining and governed system that will fund and support not only the internet as we know it today, but infrastructure capable of providing faster and more affordable internet for everyone.

Join us for our upcoming Demo Day, presented by UnifiDAO on January 13th!

AMA - Starting your Althea network with Justin Kilpatrick, Deborah Simpier, and introducing Stephen Shumaker - January 20th!

                          Discord  *** Twitter *** LinkedIn

Since our last update

Stargate upgrade complete

30 days and 96 commits later we finished migrating the Peggy module to Stargate.

Cosmos -> ETH batches functional

The Peggy system now correctly generates, signs, and relays batches of transactions for arbitrary ERC20 tokens to Ethereum.

Many many bug fixes

Using our fully automated all up testnets we run a small testnet on every code change. Advancing these tests has allowed us to patch subtle issues very very quickly.

But no amount of automated testing is a substitute for real production!

Peggy Testnet1

Our second Althea Peggy testnet is live now, with 13 validators live and still accepting more.

Lets take a quick walk from Rinkeby to Peggy Testnet #1 and back.

Download the Althea Peggy client and Cosmos blockchain software.

wget https://github.com/althea-net/peggy/releases/download/Testnet1/client
wget https://github.com/althea-net/peggy/releases/download/Testnet1/peggy 

Set them as executable.

chmod +x client peggy

Generate a Cosmos blockchain key, be sure to copy both the public address and the 24 word backup phrase, you'll need them soon.

./peggy keys add peggytest

Now let's send that address 1 ERC20 token From Rinekby to the Althea Peggy Cosmos test chain, this is a long command so you may want to edit it and then paste it into a terminal.

The provided Ethereum private key has some Rinkeby ETH on it and a few thousand Bitcoin MAXX a ERC20 token we deployed for testing. You are of course free to use any ERC20 deployed on Rinkeby.

RUST_LOG=info ./client eth-to-cosmos \
        --ethereum-key="0xb1bab011e03a9862664706fc3bbaa1b16651528e5f0e7fbfcbfdd8be302a13e7" \
        --ethereum-rpc="http://peggy-testnet1.althea.zone:8545" \
        --contract-address="0xB411f2158e70414921BEA40bC3001F89F6595F22"\
        --erc20-address="0x3A020A6A407d145de10De0367a767611F1652c06" \
        --amount=1000000000000000000 \
        --cosmos-destination="your cosmos address!"

Lets query our balance on the Cosmos chain and see those Peggy tokens! You may need to wait about 30-60 seconds for the deposit to process.

./peggy query bank balances <your cosmos address!> --node tcp://peggy-testnet1.althea.zone:26657

Congrats you now have some Bitcoin MAXX! Now let's send it back to Rinekby, generate an address with Metamask and make sure your Metamask network is set to Rinkeby.

RUST_LOG=info ./client cosmos-to-eth \
        --cosmos-phrase="The Peggy key phrase you saved earlier" \
        --cosmos-rpc="http://peggy-testnet1.althea.zone:1317"  \
        --fees="peggy/0x3A020A6A407d145de10De0367a767611F1652c06" \
        --erc20-address="0x3A020A6A407d145de10De0367a767611F1652c06" \
        --amount=900000000000000000 \
        --eth-destination="<your Metamask address>"

You should see your tokens in Metamask in about a minute. You can also view them in the ERC20 transactions section on Etherescan. Remember that ERC20 deposits are technically calls to a different contract so they don't show on the main view.

What's left before production?

Before Peggy is ready we need to complete a few more tasks:

Slashing

Slashing is at this point well planned out and mostly an implementation task. Proof submission for some conditions may be tricky to finish implementing.

Tests with full (100 transaction) batches

Althea Peggy batches transactions from Cosmos to Ethereum to save on gas. This brings the fee to cross the bridge all the way down to where it's equivalent to a simple ERC20 transaction cost on Ethereum.

Total system audit

An audit for the Solidity code has been completed, but an audit of the entire Peggy system must of course include the slashing implementation and is therefore waiting on that.

Moving forward

A big thanks to our partners at UniFi DAO for working with us on Peggy development!

Althea's goal is to launch our blockchain early next year using Peggy to help power real people using crypto to solve their every day problems.

This Saturday we hosted Peggy BrokenNet #1. Over the course of our rapid 4 hour test we

1. Successfully deployed a 10 validator Cosmos blockchain with the Althea-Peggy Cosmos module
2. Successfully deployed the Peggy Ethereum contract to Rinkeby
3. Successfully bridged ERC20 tokens from Rinkeby to our Cosmos blockchain.

We also found a couple of bugs that did not reveal themselves in our integration tests.

Validator State Updates

We were unable to update the validator set on the Peggy Ethereum contract, because it was deployed with the validator powers stored in the wrong order.

This means that while the signatures were correct, the sorting was not, and no validator set updates could proceed. This is a trivial problem with the contract deployment script, which grabs and sorts the validator set in order to deploy a ready-to-go Peggy contract pre-loaded with a given chain's validator set.

Oracle problems

While we managed to relay a couple of ERC20 transactions, the relayers pretty quickly stopped successfully relaying Ethereum events. The problem was related to the number of messages being sent. By default the http library we use to query Ethereum logs only allocates 64k of memory for the response.

The Ethereum signer component of the Peggy Orchestrator continuously spammed failed validator set updates due to our signature ordering bug. This quickly overwhelmed the request buffer and caused all Ethereum signers to fail to relay events.

Stabilizing While Developing

What we're trying to do with the BrokenNet concept is to make testnets smaller and more iterative. Instead of waiting until the module is feature complete before testing it in a production like environment and finding these sorts of issues, we can pull that development forward and test some components while we build others.

There's not a moment to waste getting Peggy into production and our goal is to do as much in parallel as possible.

What's Next?

• Support for transaction batches (Cosmos -> Ethereum transfers). ~2 week ETA
• Upgrade from Launchpad to Stargate. ~3 week ETA
• Genesis file support for chain restarts. ~3 week ETA
• First proper long running Althea Peggy testnet. ~3-4 week ETA
• Slashing conditions and limited auditing. ~6-8 week ETA

We'd like to thank Interchain and PeggyJV for providing funding and support for this highly accelerated development schedule, our developers alpe, jtremback, and jkilpatr who have provided invaluable assistance in meeting this schedule.

We're looking forward to seeing Peggy ready for production before the end of the year.

Bridges

A bridge works by minting tokens on one chain when tokens are locked on another. For example, with Peggy, you send an ERC-20 token (let’s use the Dai stablecoin as an example) to a special Ethereum address, and specify which address on Cosmos should receive it. Your tokens on Ethereum are locked. Within minutes, Peggy mints tokens for you on the Cosmos side. These are not exactly the same token as the token you locked on Ethereum, but can be considered equivalent with some important caveats. We’ll call the token on the Cosmos blockchain “CosDai” for now. But in a user interface, it would simply be referred to as “Dai”, since in most cases the difference won’t matter to the end user.

You can send the CosDai tokens around on Cosmos and use them in decentralized applications there. CosDai can be sent back to an address on Ethereum by sending them to the Cosmos side of the bridge. The CosDai is then burned, and the same amount of Dai on Ethereum is unlocked. Due to this ability to exchange the synthetic Dai, CosDai, for real Dai on Ethereum, they should trade at the same price and basically be equivalent.

This is how all the bridges mentioned above work, as well as many others not mentioned. Some, like xDai and wBTC, are custodial, in that there is an administrator or group of administrators who safeguard the tokens. Peggy is considered noncustodial because the only entities that can control the bridge are the validators of the Cosmos chain that the tokens are being transferred to, and their control of the bridge is subject to the same consensus rules as the chain. This means that the bridge is no more or less secure than the chain it bridges to.

Payment channels

This method of bridging uses something called a hashlock, which is code running on a blockchain that releases some locked tokens once the pre-image of a hash has been revealed. By lining up several hashlocks on different chains, tokens can be released simultaneously. This enables trustless cross-chain transfers of value because nobody needs to trust that anyone else will hold up their end of the bargain.

Let’s say that you want to transfer Dai (a dollar-pegged stablecoin on Ethereum) to CosDollar (a different dollar-pegged stablecoin on Cosmos). To do this with channels you first need a counterparty who wants to do the opposite trade. In practice this role would be played by a market-maker type of entity who is getting a fee for providing the liquidity. This counterparty must have an amount of CosDollar equivalent to the amount of Dai that you would like to trade/move. In this example, since they are both stablecoins, this will usually be almost the same number of tokens. We’ll call this counterparty Charlie.

Now for the move. You lock up your Dai tokens in a hashlock on Ethereum which is set to release them to Charlie once you reveal the pre-image of a certain hash. Charlie now hashlocks her CosDollars on Cosmos, to be released to you using the same hash. Once this is done, you can reveal the pre-image of this hash. Once it is submitted to both blockchains, the money will be released. This is also known as an “atomic swap”, since it is atomic- it is not possible to release one of the hashlocks and not the other. You don’t have to worry that Charlie will take your Dai and not give you your CosDollars.

This example highlights one of the biggest fundamental differences between these two methods of cross-chain value transfer. Notice that you were trading Dai for CosDollars, not a representation of the actual Dai token on Cosmos. If Cosmos does not have a native stablecoin, this whole thing is not possible. This is because payment channel swaps do not issue tokens, while bridges do.

Other details:

Bridge: Bridge theft attack

The bridge is locking a token on one side for every token it issues on the other side. This could make it a juicy target. To steal the funds in the bridge, an attacker needs to take over the validator set of the blockchain that controls it. If the market cap of the proof of stake token on that chain is a lot less than the value of the funds locked in the bridge, this could become a real possibility.

For example, let’s say that Cosmos becomes the best place for people to pay each other in Dai, due to congestion and gas fees on Ethereum. Everyone is locking their Dai in Peggy and moving it around on Cosmos. So much Dai is being locked, that the locked amount is 3x the Cosmos market cap. Mallory, an evil genius, secretly starts buying up ATOM tokens (this is the PoS token of Cosmos and determines who validates the chain). She secretly uses her ATOM supply to put her own validators in control of the Cosmos blockchain. Once she controls 2/3rd of the Cosmos validators, she can use them to submit a transaction to the bridge transferring all the locked Dai to herself. After this happens, there is a huge shitstorm and the value of ATOM falls to 0. Mallory has lost a lot of money on the ATOMs, but she has gained much more from the Dai theft.

This attack vector is not directly present with a payment channel, simply because there is no locking, and no token issuance. It may still happen though. It is a risk that is present whenever any blockchain secures value which is not its native currency. For example, in the CosDollar example, if the total supply of CosDollars was a lot higher than the market cap of ATOM, it might be possible for Mallory to use the same attack to steal CosDollars.

Aside: I’m dealing mostly with PoS chains here, but the attack described here works on PoW chains as well. Instead of buying up the PoS token, Mallory would have to buy up enough hashpower to do this attack.

Payment channels: Development complexity

Payment channels are quite simple in concept, but they always seem to require a lot of auxiliary development to make them actually work well in the real world. Their use for cross-chain value transfers is no different. Consider the counterparty, Charlie. Who is this? You don’t need to trust her, but you do need to find her. The easiest thing to do is for someone to run one node playing the role of Charlie. This might be the developers of the payment channel protocol.

This centralized Charlie can’t steal your money, but she can censor your transactions. So this is already more centralized than the bridge or the underlying blockchain.

One alternative could be some kind of a marketplace for counterparties like Charlie. Someone could help facilitate an exchange either because they want to move tokens the other way, or because they are playing the role of a market maker and providing liquidity for a small fee.

Again, the design of all this would be pretty complicated and in-depth, and is much more difficult than the basic functioning of the channel. I also glossed over some important details in my example. While Charlie can’t outright steal your money, she can keep in locked in her channel for an amount of time called the “challenge period”. I won’t get into the reasons for this and all the details, but it adds even more complexity.

Verdict: Bridges are more flexible, if you want that.

Bridges let you transact any token across any chain that they are enabled on, but you need to keep in mind that the token you are transacting is not exactly the same as on its native chain. You need to consider the security of the chain you are transacting on as well as the security of the token’s native chain. This becomes even more complicated if the token has been sent across multiple chains (this is entirely possible!).

Cross chain value transfers with payment channels are necessarily coupled with a market exchange of one token for another, and that reduces their flexibility. They give you no ability to work with a token of your choice on a different chain. However, depending on your philosophy about what blockchains are for, this may not be a downside to you. Some people believe that there should be one token per chain, and that different tokens require different blockchains. In this philosophy, transacting with synthetic tokens minted by a bridge might seem like a bad idea or a security risk, and moving value across chains with a market exchange is the natural way to do it.

Think of the contract like a highly tuned engine. It does exactly what it needs to do without a single ounce of wasted metal. But without a car built around it our highly tuned engine is going nowhere extremely fast.

For this update we have our first end-to-end working bridge flow. Validator set updates are generated, signed, packaged, and submitted to the Althea Peggy Ethereum contract.

But don't take our word for it, you can automatically build a full Peggy dev environment and run it for yourself, you just need docker and about 15 minutes.

#!/bin/bash
git clone https://github.com/althea-net/peggy
cd peggy
bash tests/all-up-test.sh

That's it! We use this same code to run an all up micro-testnet on every commit to the Peggy repo.

This command will

• build from scratch all the Go, Solidity, and Rust code.
• Deploy a Cosmos blockchain with the Peggy module installed
• Deploy an Ethereum blockchain, build and deploy the Peggy contract to that Ethereum blockchain
• Run the Peggy test Orchestrator
• The Orchestrator then bootstraps the Peggy module and the Peggy Ethereum contract by constructing and ferrying the first validator set update between the two chains!

I know this already sounds complicated, but the devil is the details. Specifically the details of signatures. Since the contract doesn't wasate a single extra cycle processing, sorting, or ordering it's inputs the Orchestrator has to make sure that everything is in perfect order.

This process is particularly finicky because it also involves the Peggy Cosmos module, which also has to store and verify signatures.

For a complete end to end validator set update you to synchronize signature generation and verification in the Cosmos module, Peggy Orchestrator, and of course the contract itself.

This includes every little detail from the ordering of the validators list down to even the smallest detail of parsing out the signature components and presenting them to the contract independently rather than in the compact form.

I'm very proud to say this all works now. It's a very important achievement, one of the hardest parts of getting a cross chain bridge working is getting everything talking to each other.

While validator set updates are the first thing to work end to end we're making rapid progress at building out the Peggy Cosmos module. Here's a short list of things that are ready in the Cosmos module and waiting on end to end integration.

• More complete (but still not final) bootstrapping process
• Various oracle operations (these will mostly be final)
• Transaction batch generation  (early version with minimum edge case handling)

In order to meet our early October brokennet target these existing patches just need to be refined and integrated end-to-end.

You'll be sending transactions back and forth before Halloween.

I almost forgot to mention, Peggy Ethereum contract gas costs are down 25% since our last update.

This is due to two optimizations a community submitted optimization on fee transfer, and a more complex change that allows for updating the validator set and submitted a transaction batch at the same time.

By allowing validator set updates and batch submission to operate independently transaction batch fees can pay to keep the validator set updated on active bridges. Even better the total cost is lower since the combined method can share the same signature verifications.

Crossing the Peggy bridge now costs less than $1 in gas per transaction using a 100 transaction batch @300gwei. Or just 15 cents in gas @50gwei.

If you want to keep up with Althea's Peggy development be sure to join our next standup and keep track of our past standups on Youtube.

We keep an up to date playlist with recordings of every bi-weekly Peggy update. In general this is the most up to date source of info about development and what's going on.

Althea builds robust infrastructure both in code and products people use every day. We are excited to join this partnership and develop reliable technologies to empower these users.

This partnership accelerates the progress of key technologies required for the launch of the Althea blockchain. We also look forward to developing blockchain infrastructure for the greater ecosystem. It is this focus on interoperability, robust engineering, and “just works” development that resonates well with our core philosophies.

The consortium is also taking a practical and iterative approach for collaboration, bringing great projects together to build core infrastructure and tools and experimenting with blockchain ecosystem cooperation and incentives in a decentralized way.

Cooperative governance and industry consortiums have, of course, been around for many years, bringing even sometimes adversarial players to the table to build or share industry resources. As we collaborate and build new things, we believe this unique partnership will be well suited to  find new ways to adapt and refine these principles specifically for blockchain and other decentralized industries.

So, what ARE we building?

Over the next few months our focus is building the ethereum to cosmos bridge, Peggy,  and Chainstarter, a service and tool for blockchains at launch to instantiate new cooperative cross chain applications.

With Peggy, we build the ability to make every Ethereum user a Cosmos user instantly,  bringing the value of interoperability, sovereignty, and blinding speed to them. We unlock defi tools without high gas fees! The Peggy bridge gives Althea the ability to use the DAI stable coin as the native transaction token (distinct from the staking token) on the Althea blockchain, which enables Althea to launch the first decentralized blockchain for telecom.

Building from this interoperable eth <> cosmos bridge, will be Chainstarter, a solution for initial distribution and funding for newly launching chains.  Using Peggy technology, any new Cosmos chain can launch initial distribution and price discovery via Ethereum’s DeFi ecosystem of automated market makers. It’s an elegant, permissionless solution for chain launch, without the burden of large listing fees.

What’s the timeline?

With the formation of this consortium and fundraise, we partnered with the amazing team at Confio to greatly accelerate the timeline and are looking to run initial “brokennet” test nets in mid-October and a target official launch in January of 2021 for both Chainstarter and Peggy.

To stay in touch with our progress, we hold regular Peggy standup meetings with updates, and you can find us on telegram.

If you want to test Peggy end to end you need to build and deploy a full software stack for two different blockchains. For the experienced this represents hours of setup work, for the inexperienced this is an unassailable barrier to contribution.

For this reason we've invested a large amount of work in test tooling for Peggy including an all up integration test that can be run on every commit.

#!/bin/bash
git clone https://github.com/cosmos/peggy/
cd peggy && git checkout althea-peggy
bash tests/all-up-test.sh

This one command will

• Build the Peggy Cosmos module
• Build the Peggy Ethereum contract
• Build the Peggy relayer and validator signer binaries
• Setup a 3 node Cosmos testnet
• Setup an Ethereum testnet
• Deploy the Peggy Ethereum contract
• Run tests across this testnet.

Right now there isn't too much to test beyond deployment of the components. But as the project progresses we plan on automating every validation we perform so that any developer can have the advantage of an instant testnet.

To access a working development environment, where the Cosmos and Ethereum chains are already running and ready for you to poke around see the

./tests/build-container.sh     run ./tests/start-chains.sh     switch to a new terminal and run ./tests/run-tests.sh     Or, docker exec -it peggy_test_instance /bin/bash should allow you to access a shell inside the test container  Change the code, and when you want to test it again, restart ./tests/start-chains.sh and run ./tests/run-tests.sh.

#!/bin/bash
bash ./tests/build-container.sh
bash ./tests/start-chains.sh
#switch to a new terminal and run 
bash ./tests/run-tests.sh
# Or this to access a shell inside the test container
docker exec -it peggy_test_instance /bin/bash 
#Change the code, and when you want to test it again, restart 
bash ./tests/start-chains.sh && bash ./tests/run-tests.sh.

We're organizing development in the Cosmos Peggy Discord channel, so please hop in and ask for tasks, notify us of open pull requests, or just chat. Code can be found at github.com/cosmos/peggy

Right now major next steps include

• Relayer logic in Rust
• Transaction batch logic on the Cosmos-sdk side
• Potential optimizations to the Solidity contract using bit-packing to store sigs
• Tests for every case where the Solidity contract should throw

ERC20 transactions cost around $4.50 at 300gwei.

At the heart of Althea's design for a  Cosmos <-> Ethereum bridge is an ultra efficient Ethereum contract. Built for the dual challenge of high fees and the large validator set of the Cosmos hub.

The Althea Peggy Ethereum contract can validate a weighted powers vote from the complete set of 125 validators on the Cosmos hub for a cost of $1.20 per Cosmos -> Ethereum ERC20 transfer @ 300gwei. For a total cost of $5.70 per ERC20 transfer.

Consider for a moment how efficient this is. 75% of the cost of your transfer across the bridge is a simple state update in an ERC20 contract. We manage to fit all the hashing and verifications for 125 signatures into 25% of the cost!

We have another two big performance improvements to go! Potentially reducing the bridge to less than 12% of the total ERC20 cost.

The Optimizations

So how did we do it?

Our team is uniquely suited for this.

I worked as a performance engineer on Red Hat’s performance and scale team for two years before leaving to found Althea.

Jehan Tremback, my co-founder and advisor has been closely involved with Ethereum since it's inception, with a speciality in payment channels.

Putting our heads together we decided that rather than trying to avoid the problem of weighted powers votes on the chain we should optimize a multisig wallet for large voting sets and frequent updates.

1. The validators list is stored as a hash: Instead of storing the validator, set the contract simply hashes it and stores the hash. Emitting an event on any change, the caller must find the event and present it to the contract. This is a 100X improvement over using a storage object for each validator.
2. Validator votes are verified in order of descending power: Once 66% of the votes are accounted for, signature verifications stop and the action passes. On average this is a 75% reduction in sig operations.
3. Transactions are batched: By sending a batch of 12 - 100 transactions, the cost of verifying the validator votes can be amortized.

Once deployed this will be the most efficient and, to my current knowledge, the largest scale multisig wallet on Ethereum.

But don't take my word for it, you can run the tests yourself.

#!/bin/bash
git clone https://github.com/cosmos/peggy/
cd peggy && git checkout althea-peggy && cd solidity
HUSKY_SKIP_INSTALL=1 npm install
npm run typechain
npm run evm &
npm run tests

As you can see here, this test is run against the Cosmos hub validator set as of 7/14/2020. The operation containing the ERC20 transfers is 'submitBatch' which is submitting 100 Cosmos -> Ethereum ERC20 transactions as a single operation.

Those of you checking my math may note that that my estimate is coming in too high. I ran my numbers using an even 5 million gas for the 100 transaction batch. On the assumption that that the EVM will require more operations to call many ERC20 contracts than the one ERC20 contract it calls for all 100 transactions in this test batch.

The Peggy Relayer

At this point, active development has moved on from the Ethereum contract.

In the current state it's possible to manually progress the Peggy Cosmos module state machine and send the resulting validator set updates to the Ethereum contract.

Current progress is focused on the Peggy Cosmos module and the tooling required to operate it automatically and most importantly robustly.

When designing Althea's payment system we ran into a lot of reliability problems with Ethereum and Ethereum full nodes [see development updates #61, #70, #71 or #73].

Trying to use Ethereum sometimes and using Ethereum all the time are very different problems.

We're bringing this experience and our battle tested tooling to the Althea Peggy relayer.

Time line

• End of August: Working relayer tooling, this means working 'valset' updates where the relayer observes the Cosmos chain, produces signatures over the validator set and ferries those to the Ethereum contract.
• End of September: Working transaction batches, the relayer will be both updating the voting set and ferrying transactions to the test chain
• End of October: Transaction batch fee market, automated testing of edgecases around bridge failure.
• End of November: Public testnet
• December-January 2021: Althea mainnet deployed with Peggy
• 2021: Peggy for the hub as part of Atom 2021

If you would like to contribute, or just keep track of Peggy development join the Peggy channel on the Cosmos Discord server!

In alignment with Althea's Cosmos based blockchain, this investment will support our work in deploying more decentralized internet networks and the work of interoperability with Ethereum and Cosmos by developing the Peggy bridge.

Cosmos's vision of interoperable blockchains resonates with our core vision of an open and interoperable bandwidth marketplace and we are proud to be a part of the Cosmos' ecosystem.

Althea makes it easy for anyone to add capacity or participate in the network. Althea’s router will always select the fastest and least expensive route on a second-by-second basis. These internet users and bandwidth providers bring a growing user base of everyday people to Cosmos, using crypto to pay for their internet.

Althea has established networks throughout the US and Africa and has seen accelerated growth since the post pandemic world has brought the need for high speed broadband into focus. We are excited to continue this growth trajectory along with Cosmos as we launch our Althea blockchain on Cosmos later this year.

“Althea is truly aligned with the Cosmos Network ethos to drive sustainability and sovereignty in our economy and society. They are a tangible example of how decentralized networks can improve people’s lives and their contributions to Peggy will be massively useful for the entire ecosystem.” said Billy Rennekamp, Grants Manager of the Interchain Foundation.

Let’s get to the fun stuff - What are we building and how do you run it?

What’s in a bridge anyway?

When we say that Peggy is a ‘bridge between Cosmos and Ethereum’ a lot of details get swept under the rug. Exactly what would this bridge be capable of? What parts do we need?

Major components for Peggy are.

• The Ethereum contract: To lock and unlock the funds on Ethereum
• The relayers: To present data about what’s happening on Ethereum to the Cosmos Zone and vice versa
• The Cosmos module: Determine the state of funds on Ethereum through the relayers + lock and unlock the funds on Cosmos
• The Client: Software run by whoever is using the bridge

For even a completely centralized Ethereum ↔ Cosmos bridge you need some version of all four of these components.

To grossly oversimplify a complicated subject all of these components exist somewhere on this scale.

In a centralized system clients simply make their contract calls and place their trust in a single manager or single key. This manager can always steal funds in the bridge.

In a multisig system a group of managers takes the place of a single manager. The multisig membership typically changes infrequently if at all. A multisig vote can always steal funds from the bridge. xDai, WBTC, RSK, and Liquid all work this way.

In a validator trusting system bridge security relies on a DPOS (Delegated proof of stake) validator set. While the bridge itself continues to act effectively as a powers-weighted-voting multisig, DPOS infrastructure allows for a much larger and more frequently changing set of voters to be practical. In addition, validator stake is now implicitly on the line, dishonesty with bridged funds may reduce the value of the validators staked native token. This is the level of security provided by standard IBC and the level of security Althea is targeting.

In a semi-trustful system, the clients trust a system of observers and the threat of slashing stake to prevent the validators from acting dishonestly with the bridge funds, even if a majority attempts to do so. This usually involves slashing on other chains. This is semi-trustful because not every client is observing both chains and submitting slashing proofs but the system depends on someone doing so. This is the level of security targeted by ChainSafe’s ChainBridge

In a trust-less system, the client itself can observe both blockchains in a highly efficient manner and confirm with some sort of succinct proof that funds are being properly handled. You may have heard this described as ‘atomic swaps’. As far as I’m aware there are no systems like this in production due to the extreme difficulty of performing these full verifications efficiently.

The overall trusting or trustless nature of the bridge depends on its weakest link.

Currently, different parts of Peggy are in progress at different levels of trust.

• The Ethereum contract: Centralized
• The relayers: Unfinished semi-trustful
• The Cosmos module: Unfinished semi-trustful
• The Client: Validator trusting (meaning a normal Cosmos full node)

Althea Peggy, foundations for the future

The plan for Althea Peggy is to

• Sync up all components at the validator trusting level of security
• Provide only bidirectional ERC-20 support (DAI to Cosmos and back for example)
• Get the above into production in a robust, stable, and efficient way by the end of 2020

This minimalist design lets us do away with laundry lists of subsystems to design and potential vulnerabilities to cover. That way we can focus on the difficult challenge of getting any version of Peggy at all ready for production use.

Our goal is to provide solid foundations for future development. A working system that can be iterated and production-ready code that can be run by people who need it today.

Over the rest of the year, we’ll be making blog posts twice a month taking you through individual components of Althea Peggy in detail. First up we’ll be showing off our highly efficient Peggy Ethereum contract.

But you don't have to wait for us to try things out, you can run the code today. Visit the Peggy repo and run 'bash tests/all-up-test.sh'.

This Oregon farm is part of an Althea network, so instead of one, siloed monthly subscription, the farmer’s Althea router selects the best upstream provider on a second-by-second basis, based on his choice of best quality or cost. For this farmer, he doesn’t need low latency for the goat cams to work well, so he sets his Althea router to prefer the more affordable rate. Then, when it’s time for gaming in the evening, he select a preference for "quality" and nearly instantly enjoys a lower latency connection.

Most of the neighboring farms in Marshland are part of this same interconnected Althea network, and many of his neighbors earn credit in the form of Dai, a blockchain based stable coin, for hosting relay antennas that connect nearby houses.

The Althea system is much more affordable to set up than legacy ISPs and can be built faster as well. Recently, in nearby Knappa Oregon, local IT professional, Stephen, started an Althea network in about 4 weeks. For years, the people of Knappa had only slow DSL or satellite internet and when Covid hit, the digital divide became even more of a life threatening obstacle. The school district here estimated that as many as forty percent of the students lacked access to broadband. The disparate population and challenging hilly topography of the area, makes the economics of legacy ISPs difficult to pencil out. Althea’s agile and resilient approach leverages private property to build quickly and affordably. We now have a fiber backhaul online, as well as about a dozen subscribers and key relays with connected neighbors. Knappa residents can now take advantage of fast internet, 50-70Mbps, for about $35/month.

Althea is not just in the Northwest either! We recently celebrated the launch of an Althea network in Enfield, North Carolina, operated by Wave 7 Communications. Wave 7 is bringing high speed, affordable internet access to an area with limited options and 250GB data caps from the incumbent telco. Althea is not only a flexible model great for subscribers, it’s plug-and-play tech that makes it easy for anyone to start and manage their network. And with the launch of our new network tools, management of routers and antennas can all be done remotely.

We’ve streamlined our process of getting started with networks, with an initial proposal, an online store, and a pre-configured “mini-mesh” kit to help get people acquainted with the system. We’ve been shipping out quite a few of those lately, and are excited about all these new networks. One of those new networks, ElWEll, looks to be live in the coming weeks, providing high speed internet for the city of Watertown, Tennessee and surrounding areas.

Networks in Tacoma and Denver had the misfortune of getting off the ground just as Covid hit, but are looking forward to installing again now as things begin to open up. The Hilltop area has almost 200 people on the sign-up sheet, waiting to be installed.

Covid hurt our supply chain and slowed the progress of our Abuja network too, but we are moving forward again, focusing on mobile users.

We are excited to announce the launch of the new network in Lagos,the largest Nigerian city, and will be working closely with the Auranetisp team as they build out a mobile-focused network throughout the city.

We are also excited to welcome new Altheahoods in Tucson and Nevada City in the United States and Tema in Ghana. We’ll also be working together with QuillaLabs & Universidad Simón Bolívar to bring affordable connectivity to the Atlantic Departament of Colombia.

The importance of high quality and affordable internet has been apparent for years, but the pandemic has highlighted just how imperative broadband is. And, until recently, building infrastructure to address the divide has involved years long planning and expensive builds. Althea’s agile model means networks can now begin building in weeks, and decentralized ownership and commoditization of bandwidth aligns incentives so networks are sustainable and grow organically.

Ready to start building your own Althea network? Check out our getting started guide here and join us on Discord for further discussion.

Let's take a trip to Marshland, OR - nestled between the Columbia river and the hills along the Oregon-Washington border. This gorgeous region is home to tall trees, sweeping farms and stunning river views. However, it is also home to people whose only option to get basic internet is to pay over a $100 per month to both satellite and cell service providers. A hefty price to pay for a crucial necessity.

The reason traditional methods for high speed internet are not available here is a combination of cost and location. Constructing a big tower is simply too expensive and you can't trench fiber into the dikes holding back the river. Rural areas like these are beyond the reach of traditional ISPs due to such geographic and economic hurdles.

Urban areas too can have their own share of hurdles - old buildings that can't be rewired, providers with monopoly on the right of way, precluding other solutions from filling the gaps, etc.

This is the story underlying poor or no internet anywhere you go, be it urban or rural. The factors contributing may differ but the end result is the same - the standard solutions are simply too inflexible.

Althea offers an alternative.

Althea changes this story.  

Althea's combination of low cost hardware and the flexible "neighbour to neighbour, pay-per-forward" model, is not only capable of operating, but thriving in places where traditional providers cannot.

To see how we accomplish this, let's look at the Althea network currently in use in Marshland as an example.

Below is the network map of our Marshland backbone link - a seven hop load balancing link between two gateways, constructed from home to home using low cost antennas.

Our first gateway (blue star on the map) is the antenna at a computer shop in Clatskanie.  Our second gateway (yellow star on the map) is the antenna at a greenhouse in Marshland.

When originally setup, these Althea gateways were islands - each with its own commerical connection, serving different networks and different customers.

With a slew of antennas (the red stars on the map), carefully aimed at each other across the landscape - we have created a bridge that allows traffic to flow back and forth, between these two islands as one unified network. Internet traffic is like water in an Althea network and it flows freely along the path of least resistance. More gateways, means more paths, more degrees of freedom and more reliability for the whole network.

This bridge not only brings internet to new customers in Marshland but provides load balancing and backup connectivity for our existing users in Clatskanie as well. They silently get the benefits of increased reliability and improved capacity as the network grows and more links are created. The network "grows" stronger. Literally.

Additionally, this system acts a distributed tower serving the whole region. We can reach almost anywhere in the general Marshland area from at least one of these antennas. The best part? It cost less than $10k to construct. Getting equivalent coverage using traditional offerings would incur around $100k in tower costs.

Althea brings together the power of high tech innovation and good old fashioned community building. It is about revolutionizing internet service to be a flexible system built by members of the community instead of a rigid system built by someone outside your community.

In Marshland, Althea goes where others could not. It's proof that local people can be a part of the solution. Building themselves better service rather than waiting for millions of dollars to be poured into stamping an inflexible solution over a local problem.

I will sign off with a beauty shot of the longest link in the Marshland - Clatskanie network that grazes the tree line and brings connectivity across this beautiful landscape.

FOR IMMEDIATE RELEASE

New Internet Service Provider (ISP) launches in Enfield N.C., offering new way to access broadband internet.

[Enfield -- June 2nd 2020] Althea, a system that enables communities to build and maintain their own decentralized internet infrastructure, today announced the upcoming build out of a new network in Enfield. Wave 7 Communications started in April 2020 is one of the first Althea networks to launch in the Halifax County area.

“Enfield residents and those living in rural areas often have limited choices for internet service, which results in slow speeds and a lack of flexible options,” said LaShawn Williamson, Partner with Wave 7 Communications.

Wave 7 Communications offers an answer. Users will receive the internet via a small antenna on their roof and a smart router, which selects the cheapest, best connection available and pays the bill - all automatically. Members only pay for what they use, so they can control the cost of their service with no contract. The average speed is 100 Mbps, with costs around $40 month.

“Our company is excited to serve the town of Enfield and surrounding area with faster and more affordable internet. We understand the importance of broadband to the economy, e-Learning, and telehealth. Our goal is to give the town a community-based option for internet service.” said Avery Williamson, Organizer at Wave 7 Communications.

Residents are encouraged to visit www.althea.net/waveseven and pre-register to show their interest. When enough pre-registrations have been submitted from any particular area, Wave 7 will begin the process of rolling out services in that neighborhood.

- Ends -

About Wave 7 Communications

Our company's goal is to provide fast, secure, and inexpensive internet service to the local community. It is also to bring economic parity through internet access to communities that do not have the same advantages as cities with broadband availability. Our goal is not to compete with the incumbent ISPs, but offer an affordable alternative to customers.

About the organizers:

Avery Williamson

Avery loves technology and seeks to bring economic parity to rural and undeserved areas where internet access not readily available. He loves his family, church, and his community. In his free time, Avery enjoys traveling and running outdoors. He also loves helping people and seeing them embrace their full potential.

About Althea (www.althea.net)

Althea was founded in 2017 with the mission to empower communities to build and maintain their own community-based internet infrastructure. This is done with technology which enables network routers to pay each other for bandwidth, letting a loosely affiliated group of individuals build a sustainable local network while monetizing their roof space.

Each node in an Althea network competes with its neighbors to provide the cheapest, fastest service to the network. This ensures that networks stay fast and affordable for consumers. Althea has networks in Oregon, Washington, Colorado, Ghana and Nigeria.

FOR IMMEDIATE RELEASE

Clatskanie Co-op is excited to announce that it is ready to expand its Althea high speed fixed wireless internet service into Knappa in collaboration with local IT professional Stephen Shumaker. The central connection to fiber, located at the Knappa Market & Chevron station, is scheduled to go live this Wednesday, June 3rd! Once operational, they will begin installing antennas to connect residents who would like to take advantage of this new service. These antennas will connect homes and businesses to the fiber backbone, and to each other, creating a resilient mesh network. Since they launched their pre-registration site about 6 weeks ago, over 100 residents of this very rural area have already pre-registered, demonstrating the demand for a better internet service here.

Althea is a system that enables communities to build and maintain their own decentralized internet infrastructure. The Clatskanie Co-op negotiated a wholesale fiber connection after  successfully hitting its target of pre-registrations from the community.

“Until now, Knappa residents have been forced to live with slow and unreliable DSL, high latency and expensive satellite internet, or 4G hotspots with extremely small data caps. The pandemic has made it clear that access to affordable, functional internet is an essential utility required to learn, work, receive healthcare and other support services, and stay connected in the modern world. Our community will finally be able to enjoy fast, reliable internet access for an affordable price,” says Shumaker.

Users will receive internet access via a small antenna on their roof connected to a smart router in their home, which selects the cheapest, best connection available (on a per second basis). Customers will get typical speeds of at least 50 Mbps downstream, 20 Mbps upstream, extremely low latency, no data cap, with typical costs around $40 per month.

Knappa area residents are encouraged to visit althea.net/knappa and pre-register to show their interest. When enough pre-registrations have been submitted from any particular area, the Clatskanie Co-op will start scheduling installations. While launching in Knappa, the service is planned to expand into Svensen and Brownsmead within a few months.

Stephen Shumaker

knappa@althea.net

FOR IMMEDIATE RELEASE:

Elwell Internet Brings Inexpensive, Reliable Internet Access to Wilson County

Watertown,Tennessee,May 29,2020-- Co-Founders of Elwell Internet, Derek Elwell and Dylan Thraen,have partnered with Althea Networks to provide faster, cheaper, de-centralized wireless broadband to residents, business owners, and schools in the Wilson County communities with little or no internet access.

Elwell Internet provides a system that lets routers pay each other for bandwidth. This allows people to set up decentralized ISPs in their communities. In the Elwell network, instead of one internet service provider at the top collecting monthly payments, many different people can earn money by expanding and strengthening the network.

Today, internet access networks are owned by centralized, usually monopolistic, internet providers. The ISP owns all the routers and networking hardware, and charges for their use with a monthly fee. In most areas, there are only one or two options. If you don't like their rates or their service, there isn't much you can do about it. Competing with them would involve starting your own ISP, large enough to be a self-sustaining business — which isn't a feasible option for most.

Elwell expects to be open for service this summer. You can go to the Elwell Facebook page to pre-register and to obtain more information.

The target audience for Elwell are rural communities in outlying areas in Wilson County that have limited access, or no access at all, to high-speed internet. These customers include residences, businesses, and school systems.

Co-Founder Derek Elwell, long time Wilson County resident and teacher: “This initiative started with the sole purpose of connecting rural students with no internet access to the world wide web. We provide a way to do this affordably with reliability. The fact that this network is a co-op makes it perfectly suited for counties like ours.”

###

CONTACT INFORMATION:

Elwell LLC

1224 Chicken Road

Lebanon, TN 37090

E-Mail: elwellnet@gmail.com

Website: althea.net/elwell

Facebook: facebook.com search “Elwell”

We've been extraordinarily busy working on COVID rapid response and the Peggy Ethereum to Cosmos bridge.

Operator tools

We've been building operator focus tools to help Althea scale. The latest versions of the Althea firmware are tightly integrated with our new management dashboard.

Once an operator is configured the router reports status infromation to an easily disgestible administrator dashboard. The dashboard even proactively detects and notifies operators of problems with links and user expeirence.

As our networks continue to grow rapidly we're working to help operators manage significantly more customers than before and deal with entierly new load patters.

The majority of traffic in our networks has shifted from night time Netflix to daytime Zoom calls. Netflix is TCP with large packets where as Zoom is UDP traffic made up of many small packets for voice and video.

Combine this change in load profile with dramatically more users and better tooling is the only way to keep track of who's having problems when and where.

Peggy

The Peggy Ethereum <-> Cosmos bridge is the key unfninished compoent for the Althea blockchain. We are working with Cosmos stake holders to produce a robust and ready to deploy design.

The existing Peggy codebase is quite advanced feature wise. But it's missing production ready Ethereum contracts to hande the Ethereum side of the bridge.

The real challenge with producing a 'production ready' bridge is successfully mirroring the Cosmos zone validator state without consuming too much gas.

We've made significant progress implementing our highly optimized bridge design over the last few weeks. We've made enough progress to confirm that our optimizations are working as expected. Validator set updates and transaction bactches both cost under 50c which is fairly impressive considering that we're just using an optimized design and have yet to do any optimization of the implementation.

We talked at a recent Cosmos governance working group meeting about our plan for Peggy in more detail and we look forward to demonstrating  working validator updates from the Cosmos Zone soon.

EIP-55 Support

We've updated our Ethereum transaction library Clarity with support for the EIP-55 address encoding standard as the default. This has been an oversight in our implementation for some time and it's good to finally get it out of the way.

Clarity is the only cross-compile friendly Ethereum transaction generation library that is portable to all 32 bit platforms. Including CPU targets like PowerPC, RISCV, and MIPS.

We want to begin by sincerely thanking all our validators and community members who have supported our project through many months!

TL;DR

• In 2019, we successfully completed the testing of our custom transaction library and Althea testnets 1-4.
• In late 2019, we began work on the design and evaluation of the "Peggy" ethereum-to-cosmos bridge.
• In early 2020, we arrived at an improved Peggy design that meets all of our needs. However that is when COVID-19 hit.
• Since COVID-19, the demand for new Althea networks has greatly increased. All our efforts have been directed there, causing some delays in our blockchain development.
• We have recently started strategic development of Peggy and reached out to Cosmos stakeholders and the Interchain foundation to support a more general purpose bridge that would benefit everyone. We welcome feedback on our proposal.

After the success of testnet chains altheazone1-4, we believed everyone had a good grasp of how to start and manage the Cosmos chain itself. We also performed testing on the transaction generation library deep_space that we will use on our routers.

While we have been quiet on the updates, we have not neglected the building and growth of our core user base - people who want faster, affordable internet! In fact, in our Oregon market we have grown by 45% since COVID-19, and have launched several new networks.

We expect to continue to grow rapidly as the need for affordable broadband in a post-COVID world is paramount. By incentivizing private property owners, Althea networks can be built faster and more affordably than legacy ISPs.

The core functionality of Althea is pay-per-forward - people get paid automatically for hosting antennas and forwarding bandwidth to their neighbors. Currently, Althea users load their routers by purchasing Ethereum with an embedded Wyre widget and a bridge converts this to stable Xdai using Uniswap.  

While this has been working for now, to continue scaling we are building an Althea blockchain using CosmosSDK. We also need a bridge from a currency with liquidity, like Ethereum, to tokens on the Althea blockchain. For our Cosmos chain this will be Peggy - an unfinished module used by a Cosmos blockchain to bridge Ethereum assets to CosmosSDK based chains.

In 2019, we launched our first test nets of the Althea blockchain with a great group of validators who helped us with deployments. At this time, we were hopeful that the Peggy module would be ready for testing and deploying our Althea blockchain in production.

In late 2019 it became apparent that Peggy was not at or near that stage and we began planning for the Althea team to complete and optimize its development.

Over the last several months, we have reached out to Cosmos stakeholders and members of the community to begin work on funding and planning the Peggy development with additional DeFi functionality.

We are excited to share with you our preliminary Cosmos Hub grant proposal and the initial planning for integrating Peggy with the Althea blockchain. We invite comments and feedback here - https://docs.google.com/document/d/17mjvH06pZglL2PkJ013xNzFjEESqHtk3nYZ66aTR9YU/edit?usp=sharing

Our preliminary road map includes testnets in the summer and fall of 2020 with a goal to launch main net in late 2020. Becasue our users use Althea as their primary home internet connection we will take every precaution to ensure all aspects of the blockchain and bridge are properly tested.

While our core focus is to support and grow our user base, we have started working on the Ethereum side of the Peggy bridge. We expect to accelerate this development and expand its scope to include more general purpose features when additional funding becomes available. We invite validators who wish to be involved with any Althea testnets to join the Althea validator channel in Discord.

Thank you all for your patience as we work towards more frequent communication. We look forward to growing the interoperable and decentralized internet with you!

Diplomats Park

Nigeria has the largest population in Africa at about 200million citizens. However, only 46% of the population have internet access via cellular carriers. Many people lack adequate and qualitative internet connectivity, This is due in part to inefficiency in coming up with innovative technological solutions that address the situation and also big telecommunication companies who would prefer to monopolise the industry and reduce healthy competition. Althea is a decentralized network which can be deployed anywhere by anyone or group of people provided there is affordable backhaul connectivity.

In Garki Abuja, Althea offers high speed fiber broadband connectivity down to the mobile phones at the best rates the country has ever seen by a longshot. It requires the installation and maintenance of radio equipment by participants and organizers which grants the community high speed internet access and lets them earn some money in the process.

Althea Diplomat Park growing day by day

Althea launched its first network in Africa in early November 2019. An initial pilot network will start with about 15 participants.

The Akanyas excited for better internet

Musa Akanya  purchasing Dai (Data Credits) by MakerDAO

The Akanyas, like every average Nigerian family, only had access via cellular data and at that had to mise the data but also use it up before the month is over. They were instrumental in acquiring the base for the gateway. A local mosque with backup power supply has allowed us to host our equipment for free. The Akanyas became the first recipients of Althea in Africa and helped us acquire 6 users which became a chain reaction to getting upto 15 installations within a short time span.

Some of the residents now stream Netflix from their Android TV and the children use Khan Academy to aid with areas of struggle in school and use YouTube to learn how to code.

Althea has also become a new source of revenue generation by some merchants in the Area. Haruna, a merchant who handles POS point of sale transfers, now sells data credits to fixed and mobile users in the area. Haruna gets more excited about internet access than selling data credits. He has become an active promoter of Althea in the neighborhood and has brought in at least 12 mobile users. Mr. Gbadebo (an Althea organizer) was so in love with the project he left Apo to Garki to help with maintenance and new installations and get his organizer fee. The larger the network becomes, the more he earns.

Yaxx and Mosque Caretaker Baba Danja

Backhaul Service Providers

Radio Backhaul Connection

Althea Gateway going live

Mr. Gbadebo and Mr Yaxx Althea organizers Diplomat Park

Haruna sells data credit for the community

Fatima, a student at Katsina State University studying computer science, became the first relay at Diplomats Park. She took a keen interest in Althea because she solves two problems unique to her. Firstly, she learns about mesh networking and is considering doing her project based on Althea as a way of getting more people in academia to learn about Althea and also to earn some money from her relay. She believes in community based internet provisioning. We also have Abdulhaqq who opted to become a relay due to the strategic positioning of his house. With a solar array for uninterrupted service delivery and a market not far away, he sees a good opportunity to beam signals from his house for affordable internet data via the Althea Mobile app and has been earning since then. His family also enjoys the service and some of them have started doing door to door marketing to get more people involved.

Fatima aspiring scientist and business woman is the first relay at Diplomats Park

First relay installation by Wisdom at Fatima’s residence

Abdulhaqq Althea’s second relay at Diplomats Park

Gbenga and Wisdom installing the Solar panels with the relay

The prices of internet in Nigeria was reviewed lately thanks to the intervention of the minister of communication and that has made people happy but are happier with Althea because it truly addresses the plight of the people living in their various communities.

Like other transaction processing platforms, similar to VISA or Stripe, it’s fiscally advantageous for the token holders of a Cosmos zone operating Peggy to leverage the liquidity locked in the bridge to collect interest. With the rise of  decentralized finance (DeFi) and, in particular, the Dai Savings Rate (DSR) this sort of high liquidity, low risk investment required is now available to everyone in the Ethereum ecosystem. Through the Peggy bridge, this functionality could soon be used by anyone with a Cosmos zone as well.

Peggy, the Cosmos Ethereum bridge, is a way to bridge arbitrary Ethereum ERC20 assets, like Dai, to Cosmos zones, or to bridge Cosmos assets like ATOM to Ethereum.

Technical overview

Peggy is a bridge made up of an Ethereum contract, which stores the full validator state and a Cosmos module run by the validators to produce updates and send transactions to and from Ethereum.

In the most basic sense, the Peggy Ethereum contract is simply a multi-sig wallet controlled by the current Cosmos validator set plus some automation. There’s nothing stopping the validators of the Cosmos hub from getting together right now, deploying a boring old multisig, and simply running a bridge by hand. Of course, voting on each deposit and withdraw manually, while possible in theory, is impossible in practice. It is the automation provided by the Cosmos module which makes a bridge a viable tool.

In that sense, DeFi investment is just another set of rules and automation for the bridge. Not a giant leap in technology, but instead, of perspective.

We propose that the Peggy Cosmos module provide its own machine-readable and automatically actionable governance proposal format for the investment of bridge capital into arbitrary DeFi constructs. For example, a governance proposal could be submitted to invest half of the bridge funds into the Dai savings rate, with a liquidation preference that $1000 at time be removed from the DSR to fund bridge withdraws if required. Once that proposal was voted in, the Peggy module would accept the new ruleset and start to execute it - balancing the investment automatically, by either withdrawing or adding funds to the DSR.


This can be further extended by adding Uniswap support. Allowing the token holders to apply their investment goals across a variety of assets that would otherwise be locked in the Peggy Ethereum contract.

Finally we don’t even have to limit ourselves to a pre-defined series of investments, the governance proposal itself could define the required contract calls to deposit, withdraw, and monitor the balance of a proposed investment.

While it will remain possible to construct an Ethereum contract that the Peggy Cosmos module couldn’t interact with without a module upgrade, the vast majority of DeFi contracts now and in the future would be only a governance proposal and a vote away.

Althea’s Trustless Peggy Optimizations

We envision this Defi value capture in the context of our improved, trustless Peggy design.

The proposed security model of the Peggy Ethereum bridge uses a mix of semi-trusted parties and game theory to overcome key difficulties with the Ethereum execution environment. The main difficulty being updating the validator set on every voting power change and performing signature verification across all validators for each transaction is inordinately expensive in terms of Ethereum gas.

Althea has developed a series of optimizations that allows for the cost of validator setup updates to be reduced to 12c every 24 hours on average and for the cost of transactions to be reduced to simply the cost of the underlying ERC-20 send given a sufficiently active bridge.


These Ethereum contract optimizations allow for a simpler bridge design and a focus on adding extra features like the DeFi support. Working together with Interchain and Cosmos stakeholders, we hope to work over the coming year on implementation. Public tests nets are estimated in late third quarter 2020.

About Althea

Althea builds tools which allow routers to pay each other for bandwidth using micro-payments.

We specialize in producing reliable, highly optimized code that operates in harsh environments, so we are well suited to building this functionality into the bridge. Justin Kilpatrick co-founder and CTO worked as a performance engineer on Red Hat’s performance and scale team for two years before leaving to found Althea. Jehan Tremback co-founder and advisor has been closely involved with Ethereum and the programmable blockchain space since its inception. He scoured the earth for Solidity optimization tricks to create Guac. (Guac is a highly optimized payment channels implementation for Althea. It is the most optimized non-hub payment channel library available.)

To address the immediate need for more broadband infrastructure we have put together a Rapid Connectivity Response. This is a plan to build a 100Mbps, affordable broadband network in as little as two weeks. This method can be deployed in both rural and urban environments to rapidly build or expand a network.

Althea networks are adaptable, locally based and can be deployed rapidly. Traditional ISPs struggle to make the economics of providing connectivity to rural communities work, and build slowly - often taking years to build a network. By using private property, Althea networks lets communities build networks in an agile way, to affordably and quickly build effectively to areas with a smaller, more spread out population. Everything is plug and play and billing is automatic, so the network is easy to set up and maintain.

Rapid Connectivity Response

Step 1: Assessment (2-3 days)
FCC and other reported data is inaccurate and often does not give a complete understanding of exactly where there is a need.

• Reach out to local school board and educational leaders who can give a quick perspective into the needs of the community. County commissioners, local city mayors and managers, and state representatives may have already identified “pain point” areas that need networks. Starting a Facebook group or page which solicits the voices of the community can also be a great place to get started.
• Locate a local technical help for the management of the network. These local network operators are automatically paid a monthly fee through Althea’s software and we provide full tier one tech support for operators. This can often be some one already performing IT, camera installs, or computer services in your area. Althea is very easy to use even for someone with no prior knowledge of networking. We provide education, tier 1 support and dashboard tools for the network operator.

Step 2: Planning (1 to 2 weeks)

• Once an area has been identified, we set up a sign-up website, like www.althea.net/clatskanie. Interested subscribers can sign up, and get easily routed to a calendar app to sign up for an installation or a call to answer questions. This becomes an easy way to manage new potential subscribers and get the word out about the new network.
• Finding a wholesale connection can often be the most challenging part of starting a network, however, there are quick options that are based on using resources that already have connection. We can find the creative solutions that are the best fit for your community.
• A public building like a library, school, university, or city hall. Working together with the appropriate contacts you can build out a sustainable solution that can provide some revenue and utilize public space and bandwidth capacity. A public wifi-hotspot can be built alongside the network, providing immediate resources to people needing to access the internet. This may be a slower option, because in many cases public entities are overwhelmed with other aspects of the crisis.
• An on-net business. Many businesses are experiencing slow downs and extra income from roof lease or providing free internet can be a welcome proposition. Contact the wholesale department of the fiber provider to negotiate a rapid provisioning of the connection for wholesale use. For an “on-net” building this can often be as little as 1 to 2 weeks. Building of the main infrastructure can take place while the service order for wholesale connection is moving through the system, so you are ready to provide subscribers with broadband as soon as the network comes online.
• Current ISPs. Often it is not economically viable for the incumbent ISP to service an area using legacy methods, but they are happy to grow their network using Althea’s unique model. This can be a win-win situation for everyone involved and many ISPs are happy to discuss how this could be arranged.
• Getting equipment and installing. Together with our consultants, we can plan, engineer, and source gear for your network. Most household equipment can then be purchased at buy.althea.net by subscribers, and shipped pre-configured to the network operator. Because these networks can be built from “neighbor-to-neighbor” instead of large towers or along public right of way, the system is easier to maintain, and affordable and quick to build.

At this point, you have a working network that can start quickly and grow sustainably. This pandemic has presented us with great tragedy, but it also has brought us an opportunity to come together to solve the digital divide in a lasting way.

With the rapid increase of e-learning, telemedicine and work from home the Covid crisis has exposed just how deep the digital divide is. Despite billion of dollars of federal grants and years of work, big telecom has still failed to make the economics of sustainable development in rural areas work.  

There is a better way forward, and we should start now. Communities can rapidly build sustainable solutions to meet the needs of even remote areas, so that people can learn, work and receive healthcare from home.

Contact us here to get started and receive a custom plan and getting started guide for your area.

A couple of weeks ago I got a very small box in the mail. An 'airCube ISP' a new $30 home router offered by Ubiquiti. I unboxed it and about two hours later it was running Althea.

Phrased that way it sounds underwhelming. Lets dig in a little.

• Jailbreaking the firmware using the OpenWRT wiki. Ubiquiti signs each firmware with their RSA key, but by loading a specific config we can defeat the check.
• Identifying and downloading the correct source code to build a x86 -> MIPS cross compiler. Required to build code that will run on this devices processor.
• Cross buildng the Linux kernel and userspace for the MIPS cpu architecutre.
• Cross building the Althea billing software and assembling specific tweaks.
• Packaging all of this into a highly compressed image clocking in at a hair below 10mb.
• Flashing this image to NOR flash. Unlike the NAND flash you're used to NOR flash takes minutes to write, but is more robust for long periods of reads.

All of this was done via a script. Most of the two hours was sitting around browsing Reddit waiting for the system to finish compiling. This sort of tooling is increadible, it's easier to build a whole operating system from scratch for a random embedded device than it is to sort out the Javascript dependencies to build some web applications.

Now that a config has been added to our firmware builder new images and updates for this device will be produced automatically. When it comes time to deploy an update or build a new version I won't even need to consider this specific device. It just happens.

I chose the airCube here for a specific reason. It's very cheap, turns out it's processor is too weak for us to use in the developed world. So we're testing out how easy it is to order in the developing world.

It cost Ubiquiti millions of dollars and years to develop this hardware and setup this supply line, their hope is that the Ubiquiti backed management software on this device will tie more ISPs into their platform.

In the ISP world appliances are king. Networks are typically bespoke for each ISP and the complexity of making general purpose appliances combined with the low risk tolerence of ISPs mean that most networks are dependent on specific vendors for each class of appliance or component.

There is no app platform for managed switches or routers. Either the manufactuer provides a protocol or your done.

Once locked into the Ubiquiti ecosystem buying Linksys consumer routers would cripple an ISPs ability to manage those customers and potentially not even work properly if they don't provide the right features to tie into your specific networking architecture.

This is why Althea's approach is as radical as it is obvious. Routers are, for all their specific quirks, just computers. Like your desktop, phone, or a server.

All of those can run huge libraries of software via Linux, Windows, or Android. Ultimately open platforms eat any market in which they are available, because the value to the customer of flexbility is unquestionable.

These past few weeks have been busy, technical and otherwise. But here's what I've been thinking on.

Our latest release Beta 11 contains a lot of new features and set a new record for the number of release candidate builds required to get it stable (11 to be precise).

But I'm going to focus in on a specific problem. A four car pileup of complex codebases, starting with static binary building on Rust.

A static binary has no dynamicly linked libraries from the local machine. Meaning that while it depends on features of the operating system to run all of the code it needs is carried with it.

This is a key distinction, a static binary for Linux is still useless on Windows. But a dynamicly linked binary is tied to a specific version of whatever software it is linked against.

In our case the problem is at the intersection of Rust, MIPS, and soft floating point.

❯ find build/ -name rita | grep "sbin" | grep "staging_dir" | xargs file
build/staging_dir/target-arm_cortex-a9+vfpv3_musl_eabi/root-mvebu/usr/sbin/rita:         ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV), statically linked, no section header
build/staging_dir/target-arm_cortex-a7+neon-vfpv4_musl_eabi/root-ipq40xx/usr/sbin/rita:  ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV), statically linked, no section header
build/staging_dir/target-arm_cortex-a7+neon-vfpv4_musl_eabi/root-brcm2708/usr/sbin/rita: ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV), statically linked, no section header
build/staging_dir/target-mips_24kc_musl/root-ath79/usr/sbin/rita:                        ELF 32-bit MSB pie executable, MIPS, MIPS32 rel2 version 1 (SYSV), dynamically linked, interpreter /lib/ld-musl-mips-sf.so.1, no section header
build/staging_dir/target-mips_24kc_musl/root-ar71xx/usr/sbin/rita:                       ELF 32-bit MSB pie executable, MIPS, MIPS32 rel2 version 1 (SYSV), dynamically linked, interpreter /lib/ld-musl-mips-sf.so.1, no section header
build/staging_dir/target-mipsel_24kc_musl/root-ramips/usr/sbin/rita:                     ELF 32-bit LSB pie executable, MIPS, MIPS32 rel2 version 1 (SYSV), dynamically linked, interpreter /lib/ld-musl-mipsel-sf.so.1, no section header
build/staging_dir/target-x86_64_musl/root-x86/usr/sbin/rita:                             ELF 64-bit LSB executable, x86-64, version 1 (SYSV), statically linked, no section header
build/staging_dir/target-i386_pentium4_musl/root-x86/usr/sbin/rita:                      ELF 32-bit LSB executable, Intel 80386, version 1 (GNU/Linux), statically linked, no section header

Notice how the MIPS binaries are the only ones that are not statically linked.

Part of the problem here is that Rust doesn't have a 'build a static binary' flag, Go is famously universally staticly build. Meaning it will always build portable binaries. Rust on the other hand gives you the option of building static binaries. But that option and exactly what else impacts it isn't exactly clear. Other than one rather cryptic book page noting that you should use the MUSL libc implementation.

Using MUSL is not a guarantee, it's a polite request. For whatever reason the Rust soft float flag does not make it's way to LLVM for MIPS targets and we end up with.

build/staging_dir/target-mips_24kc_musl/root-ath79/usr/sbin/rita:                        ELF 32-bit MSB pie executable, MIPS, MIPS32 rel2 version 1 (SYSV), dynamically linked, interpreter /lib/ld-musl-mips-sf.so.1, no section header

Between Beta 8 and Beta 11 the OpenWRT was updated and the calling environment, specifically this floating point library, is not backwards compatible.

Binaries built against the old library work when run against the new library. But binaries built against the new library (Beta 11) don't run against the old library (Beta 8).

For this update we can hand build binaries against an older libc and ship those as special versions of Beta 11.

But how do we want to deal with this going on into the future? We do want to upgrade our MIPS building toolchain someday.

We could contribute to upstream Rust and see what it takes to get a fully static binary, or we could move in the direction of updating more packages on the local devices. After all if we ship a new libc and kernel update things should work just fine.

But that's not quite as simple as it sounds. In this act we bring in the fourth complex nuanced system in this dance. The OpenWRT flash storage layout and filesystems pages are very educational here.

Cheaper MIPS routers typically use cheap and very small NOR flash chips. Sized 8MB, 16MB, or 32MB.

In order to better deal with this tiny amount of storage OpenWRT uses a dual filesystem setup. SquashFS a highly compressed read only file system + JFFS2 for writeable storage.

SquashFS throws out everything, including the kitchen sink, in order to make files smaller on the disk. But you can never modify it after the initial flash.

This brings us back to our last issue. If we try and ship a kernel and libc update we're going to double the storage size for both items. We will have the old version in SquashFS, which we can't erase without reflashing the device (risky to do remotely) and the new version in JFFS2.

At this point I've written a lot of words on this subject but I haven't really made an engineering decision on what solution to pursue. This problem is nuanced and occurs across several domains, which is part of what makes it interesting. The easiest thing to do is figure out how to build static binaries, but doing things like kernel updates remotely on these devices isn't a task that can be ignored forever.

In other development news I've made a few safety and performance patches to the main Rita codebase, the rest of my time is dedicated to the development of the nascent network operator tools codebase. The product of which I hope to reveal in a month or two.

Author: Justin Kilpatrick <justin@althea.net>
Date:   Wed Mar 4 15:21:32 2020 -0500

    Use liboping for ping actions in althea_kernel_interface

    This change pulls in the c library oping and a Rust crate providing
    bindings.

    This lets us perform ping operations without shelling out and most
    importantly control the ping timeout with a resolution higher than
    one second.

    The one second timeout minimum is a key reason why the exits endpoint
    can run slowly. As it serially pings all the exits and the inside
    exit tunnel ip for the registered exit.

    That's at most (Num Exits + 1) * Timeout seconds of waiting time. This
    is actually a pretty rare case that all exits are inaccessible AND
    we have routes to them in Babel. But sadly this rare case is most commonly
    when people are looking at the dashboard! Strange connection troubles, bloat
    etc etc. So it has an outsized impact.

    This change uses our new found freedom of the timeout to reduce the exit ping
    timeout to 200ms. Which is pretty conservative and helps with the maximum wait
    time.

    Perf Check:
    previous common case: 1s
    previous worst case: 10s
    new common case: 900ms
    new worst case: 1.6s

    So ultimately
    +1 for less shelling out
    -1 for more C code
    +1 for more performance

Author: Justin Kilpatrick <justin@althea.net>
Date:   Tue Mar 3 17:18:41 2020 -0500

    Forbidden settings merge values

    We're not working to give network operators more direct access to the oracle
    features. So it's important that it be a safe environment. While it's impossible
    for us to read the eth_private_key remotely from any router the settings merge json
    does provide a chance to overwrite it.

    This really isn't acceptible froma saftey perspective. This patch implements a small
    but effective blacklist of settings values where local router state is stored in such
    a way that remote updates can't possibly be anything but destructive.

In the US, 65% of consumers get their internet from just two cable providers. These providers charge too much for slow service and leave behind many Americans who can’t access the internet due to cost. But what if anyone could start or grow a network? What if we could take back our choice, control, and ownership of the internet? Over the past few months, Althea has helped make that dream of ownership and choice a reality for Americans in rural Oregon, Tacoma Washington, and Denver, Colorado. More Althea networks are in the works as well - in Washington, Oregon, North Carolina, California, Pennsylvania, and many other areas. Althea’s automatic routing and billing make it easy for anyone to start a network or grow an existing one.

We are excited to celebrate the on-going success of our network in Abuja, Nigeria. Yakubu Yakubu, the network organizer, has done great work setting up the initial network of about 16 subscribers. People in Abuja had been paying very high prices for mobile only internet. Now, people can install Althea routers in their homes, pay affordable rates, and use the wifi-hotspot to sell internet to nearby mobile phones. In fact, in just a few weeks, we now have about 20 people using the Althea mobile app for their internet usage, and growing fast! Yakubu, or Yax, also just installed the first relay client, who is helping to expand the infrastructure of the network and make money for hosting extra antennas. The network continues to expand as fast as we can get more antennas shipped in.

Following the Abuja network, a new Althea network in T’adi Ghana came to life in February as a small existing WISP changed over to the Althea system. By empowering local business people to own hardware and sell to their mobile-only neighbors, the network can grow organically and be a better revenue source than a traditional WISP. Getting routers and antennas on location, continues to be the biggest difficulty in both Ghana and Nigeria.

We have been working hard to recognize the roadblocks and hurdles to starting a network and developing services to help smooth out barriers and support new and growing networks. For example, subscribers can now buy equipment with a six month payment plan at buy.althea.net, and relays and new networks can buy gear there too. We also have a new, ready-to-install “mini-mesh” package available if you are looking to try out Althea or set up a small network. Digital services, like Sling, will soon be available through website links too, so subscribers can easily sign up and “cut the cord” of their existing cable providers. We help support new networks through every part of the process, from marketing to sourcing backhaul and technical support.

Our latest release features the implementation of the Wyre widget, creating a smooth way to add-funds to the router. For the user, this feels very similar to using a debit card as you normally would to buy something, abstracting away most of the cryptocurrency aspects. This is also a benefit over using the Coinbase wallet, whose interface references investor language like “portfolio and trade”.

One of our main development focuses of the coming months will be a better developed network organizer dashboard and tools to help operators manage their network. This will be comparable (and most likely better :) ) than Ubiquiti’s offering, UNMS, and will make it easier for WISPs to integrate Althea into their existing network.

The other development focus will be to begin integration and testing of our Althea blockchain built on Cosmos. This important component will allow subscribers to use the stable currency dai to pay for their internet, while transactions are processed quickly and securely using a robust proof-of-stake blockchain.

The core of the Althea system is pay-for-forward, which allows anyone to seamlessly add capacity to the network, and the price-aware routing protocol which chooses your upstream carrier dynamically based on user choice of price and quality. Instead of the locked-in monopolistic system, Althea imagines one open network made up of infrastructure with different owners that automatically routes traffic and compensates participants. We no longer have to accept the legacy system of the lowest throughput at the highest price - Althea networks can start and grow anywhere. Is it time to start your Althea network?

Go to www.althea.net to learn more.

With the new Beta 11 release users can now use the Wyre widget to buy and send funds directly to their router.

We've been dreaming about a system like this for a very long time. While Wyre pay isn't available in every state or region where it is available users won't have to navigate Coinbase any longer.

The process of buying and then withdrawing to the router was essentailly our largest usability barrier.

I'm really looking forward to seeing this in production over the next few weeks.

Bugfixes and optimizatons

As we scale up our operations new bugs continue to become apparent. This is just a small selection of recently fixed bugs.

commit 9ad66361da8b6dd9b3372ee64c17361281fd1e3d
Author: Justin Kilpatrick <justin@althea.net>
Date:   Wed Feb 5 14:31:15 2020 -0500

    Reduce unneeded database loads in status request path

    The exit was performing a db request to check if a user entry existed
    and then just a few lines later requesting that same entry. We can save
    the round trip time and grab it while we check if it exists

commit 2bda1b3b232e387c70ad90e405a749ae8d63a19e
Author: Justin Kilpatrick <justin@althea.net>
Date:   Wed Feb 5 15:06:43 2020 -0500

    Optimize client database entry updates

    Previously several database requests would be made for every client
    checkin to potentially update the email and phone number.

    Now we only perform an update when there are actual changes and also
    only update the last seem timestamp every 12 hours.

    This required banning entry timeouts less than 24 hours but that's
    a satisfactory compromise for much faster general operation.

    The total number of database requests per endpoint hit should be
    reduced from 5 to 1 on average by this patch and the previous one.

commit ee8169d3d8975982d101fbe301379dadddbab856
Author: Justin Kilpatrick <justin@althea.net>
Date:   Thu Feb 13 21:31:45 2020 -0500

    Correct payment validator logic to prevent double payments

    Previously we used independent do_send statements in order to validate
    payments. This is logically incorrect as it's possible for the payment
    to be accepted in debt keeper before it is removed from the validation list
    in payment validator.

    This patch corrects that error by making sure that in order to send debt
    keeper messages the removal from the existing list must first succeed.

    The next race conditon to consider is that multiple calls to remove
    may be running in parallel. We address this  by having remove return
    an error when no such transaction exists. Since Actors themselves
    serialize access to self this means removal from the list is fully
    atomic.

commit e8fa2fa35e255b6f706b21c9acfbd00a1e0b17bb
Author: Justin Kilpatrick <justin@althea.net>
Date:   Tue Feb 11 07:48:25 2020 -0500

    Add protection for full node communication

    After discovering that nodes would overpay the exit I've been looking
    to prevent it from happening again comperhensively.

    My first attempt was to refund incoming transactions immediately and
    add long timeouts to the transaction checking. While refunding incoming
    transactions has worked quite well to unwind our overpayment balances
    long timeouts has served it's purpose but caused other problems.

    Like a generally huge number of timeouts in the common loop.

    This patch is a replacement for the long timeouts patch where instead
    of making very long lived futures we simply keep track of if we have
    actually talked to a full node about any given transaction. If we
    have not succeeded we panic.

    Both servers and clients have an auto restart catch. In some cases
    this won't help, in others where the node is merely overloaded it will

    In the specific case of the exit running slowly and not validating anything
    this would resolve the issue nicely.

commit 43222e762f17aaa511534fb246fad5be079cb2b6
Author: Justin Kilpatrick <justin@althea.net>
Date:   Sat Feb 8 22:01:05 2020 -0500

    We set the average Xdai gas amount

    Since we make up such a large porition of the xdai transaction
    volume we effectively set the gas amount you'll get from the
    full node endpoint.

    Therefore to actually lower our own gas costs it's not enough to
    remove the constant, we must also enforce a maximum gas price.

commit eb2f4282fa467be7c47ece19835ca6de6437beb1
Author: Justin Kilpatrick <justin@althea.net>
Date:   Mon Feb 10 13:36:45 2020 -0500

    Cache most exit vars

    This patch caches all the exit settings referneced by the endpoints.

    As exits scale more and more threads get thrown at their threadpools
    resulting in rising and eventually untenable lock contention on the main
    SETTINGS lock.

    Most things behind that lock we never really want to change at runtime.
    other things like the price we do.

    This patch isn't the best soltuion to the problem by far, moving the locks down
    to a more granular level within SETTINGS is the real solution here. But that would
    have wide reaching syntax changes throughout the codebase. No to mention a re-write
    of settings serialization as you obviously can't write locks to the disk.

    Overall I find this to be an acceptable compromise.

    Price is the only thing we must update at runtime so it is refreshed from the
    main settings object every 10 minutes. This will block one of the many worker
    threads while it waits for all other worker threads to release their read locks,
    updates the value and moves on. This only requires a single read lock on the main
    settings and won't block the main thread. Two big advantages.

    Remember actual exit billing is synced with the clients through endpoint hitting
    so this won't affect payments. But it will create slight discrepencies in the user
    local accounting since local billing won't update the updated price change.

I'm actually still in the progress of debugging a dns related issue on the exits. Hopefully our next dev update can cover that in detail as it's been an interesting ride.

It's been nearly a month since my last development update, sorry about that, it's been an incredibly busy time.

Without further delay, here is the Althea for Android app. You will need to use it with Beta 11 RC6 or later of the Althea router firmware. You can find a list of supported routers here.

The app is on xDai by default and I'll add a menu to change that at runtime when I get a chance. Or you can add it yourself here. Like everything Althea, the app is Open Source and Apache 2 licensed. It's missing some bells and whistles, but it works in a robust and secure way despite all of the roadblocks Android put in our way.

I believe I could dedicate a whole series of blog posts to why Android is a fun platform when it comes to writing network applications.

• On Motorola phones fe80 IPv6 addresses work to contact routers without the user requesting to 'stay connected' to the WiFi. But IPv4 does not
• On Samsung phones fe80 IPv6 addresses don't work at all, but you can always use the IPv4.
• On Android 8 and 9 the response to the boolean function is_any_local_address() is the opposite for the same IP.
• You're really better of comparing bits yourself to check what subnet an IP is in.
• But haha Java has no unsigned bytes, 192 is -64
• When trying to read the current WiFi SSID name, some phones return null when WiFi is disabled, others return <Unknown SSID>. Even better they will continue to return <Unknown SSID> even when connected if the network does not provide internet.
• The same function call on three different phones can be the key to everything working on phone A, do nothing on phone B, and crash the app on phone C. No - I'm not even talking about the documented API changes where they at least tell you this will happen.

And many other fun things I have blocked out in order to maintain my continued sanity. Compared to all of that, the router side changes were a breeze. The addition of a very small billing module and some minor modifications to the usual neighbor tunnels was all that really needed to happen.

What's new in Beta 11

• Support for Althea android phone clients
• Fixes for a bug where a node may run slowly and fail to validate all payments indefinitely
• Fixes a bug where clients could build up overpayment credit with exits without using it
• Fixes a bug where the clients would fail to bill correctly if the exit failed to respond
• Easy exit adding interfaces now in the dashboard
• Refactor of interfaces support to simplify WiFi port toggling
• Support for Multi-Collateral DAI
• Fixes for non-domain name manual peers
• Dependency update
• Usage history storage is now compressed for a 85% reduction in disk space usage
• Start migration to rust std futures

Overpayment Settling

When a router is paid too much by another router it puts the extra into a field called 'incoming payments' for later processing. This doesn't happen very often but I became interested what exactly was in this field while trying to clean up the debts storage files.

Routers have very limited disk space and I figured if we didn't owe anyone anything we could safely delete the entry. Like anything to do with billing, especially involving deleting data, I decided to inspect this assumption very carefully.

Incoming payments could typically get cleared out when a router is starting to get into a low balance state. Its overpayment credit is then applied by the other router before it's actually considered out of funds. Usually this is just a few cents at most. Probably due to some sort of crash or network disruption.

After more investigation I found that a small contingent of users who had always been very diligent in depositing funds into their router (never letting the balance even get close to low) were owed several dollars by their billing peers due to errors built up over the last year. Beta 11 contains a patch that will ensure these funds are applied right away so that it's no longer possible for them to bill up. These diligent users will see this as a few dollars of free service.

Overloaded Exit Fixes

Routers compute local bills for forwarded traffic, but for client traffic (download) they compute both their own bill and ask their exit server what it thinks the bill should be. This is important as it's possible for the exit to send more data than the client can see due to packet loss.

The Lagos exit in our Abuja deployment had been running rather slowly. Combined with the sudden addition of a dozen users this resulted in a short period where the exit was unable to respond to users requests to the exit. This effectively halted billing until the alarm was attended to and the exit repaired. While fixing this bug I couldn't help but think that the clients should have just gone ahead with their own estimation rather than doing nothing.

In Beta 11 routers will fall back to local billing only in the absence of an exit response to their settle-up request.

The major hudles where as follows.

1. Turns out Ipv6 multicast sockets need to be rebound on all network change events AND on a timer in order to work consistently.
2. Getting Android to route your app over WiFi networks that aren't detected to have internet is possible in a limited since that's good enough to complete (1). But not good enough to go back to our original design.
3. There are 8 possible connectivity states between the phone and router. Handling every one correctly took some doing.

I'm glad to report that everything works as expected and after some tweaking It's quite the smooth process to turn on and off.

The next major implementation step is to get into the billing code. I'm significantly less concerned about this portion of development because there aren't going to be a huge number of suprises to shake up the time line. Having a phone send transactions and act as a cryptocurrency wallet is something that lots of peole have done before and also something that doesn't need to interact with Android's restrictive network stack in any non-traditional way.

I hope to have a version with working billing out by mid January. This will be a generally usable Alpha build and at that point I will make a call for testing.

Multi-Collateral DAI upgrade

We successfully updated the routers in our various networks to MCD or 'multi collateral DAI'. The DAI stable coin works to maintain it's $1 = 1 dai exchange rate by using colateralized debt positions and up until now these positions where exlcsivley held in Ethereum. With this upgrade that is no longer the case and DAI is now backed by a mix of currencies.

Credit to the POA bridge development team for making backwards compatiblity happen with their bridge upgrade and saving us from a risky holiday deployment.

It has been exciting to see the fulfillment of a vision in Clatskanie, Oregon. We’ve watched the network grow “block by block” as new relays connect their neighbors, and those neighbors begin connecting their neighbors. When I first started the network, I worked to get the word out and sold Althea as faster, cheaper internet made possible by neighbors selling neighbors bandwidth.

This was a simple proposition and one that resonated well with people with a 1Mbps connection. But it also caught on with many people with decent service too, because Althea gave them a local option and a way to make passive income.  As the network grew, I noticed that people took ownership of their connection, that it was more than just great internet service. People running relays began to go door to door to sell their neighbors on bandwidth, and the network started growing organically.

We are finishing out the year with over 50 subscribers in Clatskanie, and we will be adding another high speed, fiber backhaul location, an Althea gateway, accelerating the growth to new areas. As these areas spread out, they will interconnect, giving subscribers multiple upstream providers. Althea’s price aware routing protocol will instantly switch between the faster and cheaper connection, giving people the best connection possible on a second-by-second basis.

During the year we made the switch from the Ethereum blockchain to the stable xDai currency. In most cases, subscribers still buy Ethereum using the Coinbase app and send it to the router, then the Ethereum is converted seamlessly to xDai using a Uniswap bridge. Relays and organizers who make money from the network withdraw in Ethereum using Coinbase too, and all transactions are in xDai so the price does not fluctuate and transactions are processed quickly. In early 2020 we will be implementing a Wyre widget directly on the dashboard that allows subscribers to simply use Apple Pay to load up their routers.

In August, we partnered with Republic to launch the first ever legally compliant security airdrop open to everyone. Aspiring network organizers, who perform the role of customer service and management of the network, earn tokens for learning about Althea and starting a network in their city. We now have over seven thousand participants! Twenty of these participants have started the process of gathering interested users through a pre-registration site like www.althea.net/tcn  and two of the networks have already broken ground in Tacoma Washington and Abuja Nigeria.

The launch of the Tacoma network in mid-November is an excellent example of the advantage of leveraging private property and the power of neighbor-to-neighbor marketing. After initially launching their pre-registration site in July, online marketing and PR generated a buzz and pre-registrations in a spread out and sporadic way. But once the gateway (connection to wholesale fiber) at David’s house was established, David and the organizers went door-to-door, and nearly half of the block signed up!

The Tacoma network also features 60GHz, or 5G, antennas that provide fiber-like speeds at prices of around $35 dollars a month for most households. These millimeter wave antennas must be placed in line of sight and within a few hundred feet of the user. Incentivizing homeowners to re-sell to their neighbors allows Althea networks to use this technology more efficiently than cellular carriers who must pay for right-of-way on power poles or city lights for small cells. As new relays come on line in Tacoma, we will see the network spread out and grow - one high speed connected neighbor to another.

In December we also celebrated the launch of our first African network in Abuja, Nigeria. Like much of Africa, internet speeds in Abuja are slow and most people only have access to internet on their phones. Since the launch of Althea, there has been excitement over faster and cheaper access and many people are looking forward to getting connected. Early on, there have been a few setbacks getting affordable antennas and routers to the organizers there, but waiting demand and the launch of the wifi-to-phone capability promises to bring a fast growing network in 2020. This wifi-to-phone functionality will allow anyone to use their Althea router’s wifi connection to connect and sell to nearby mobile phones, increasing the revenue for relays and broadening the market. As Althea CEO Jehan Tremback said “Any place within range of an Althea router can be an internet cafe - you just need the coffee.”

As 2019 draws to a close, we want to thank all of our Althea organizers, subscribers, test validators and supporters. This year was a foundation year - we have set the stage, the lighting and props are ready, and the actors know their lines. In 2020, it’s curtains up and time for the networks to launch, grow and flourish. From major cities to small towns, Althea networks will spring up, and like the inevitable forward movement of a tidal wave, networks will grow as neighbors connect neighbors. Legacy monopolized ISPs will be a thing of memory, encompassed by this tidal wave of growth they will become relays in Althea networks as well. The marketplace will be fundamentally altered, and Althea will become the way all bandwidth is bought and sold.

Recently we've been working on a phone client for Althea. Our first design was to have every phone as a Babel node, advertising it's own routes and generally functioning independently.

It turns out Android will let you do this, but only in a very specific way that's not useful to us. Hopefully someone else finds the knowledge useful.

The restrictions

1. Android does not allow you to assign or modify network addresses unless you are inside a vpn tunnel. So you are at the mercy of whatever routes it decides to install into the routing table for a local network
2. Android does not support DHCPv6 in an attempt to force ISPs to assign IPv6 address blocks to customers rather than using nat6 or some other horrible solution, this means that even though we control the router we still can't insert routes, just addresses via SLACC.
3. Android only allows sufficient routing to contact immediate neighbour devices (such as to authenticate with a captive portal) until it has successfully completed a DNS lookup and https handshake with a Google server.

The tl;dr of this is that you can have an IPv4 lan with Android but not an IPv6 one, IPv6 routing can either reach Google or doesn't work at all.

As you can imagine this was a rather frustrating setback, requiring us to drop about a week of proof of concept work. Including a fully functional Babel light client. Which as far as I know is the only somewhat functional implementation of the Babel spec on Android.

Ok, so what's the workaround?

There are three possible paths forward for Althea.

• no boostrapping network, this would be like Airport WiFi, a little screen then internet. Everyone else on the network can dump your traffic and most importantly it's really easy to spoof anyone else and stick them with the bill. So that's a no go.
• Bootstrapping network with IPv6 link local addresses. This is possible but only with some sort of address translation layer that's like NAT for IPv6 but even worse and more complicated. No thank you.
• Bootstrapping network over IPv4. This is the one I ended up going with.

IPv4 light clients

This is the new design I've come up with to overcome Android's short comings, have phone clients act as a sort of light client to the host router.

The host router can then share it's IPv4 addressing to the exit and through the exit to the outside world.

This design is somewhere between the captive portal you would find at a Starbucks and the fully secure solution you find in the routers.

By using a WireGuard tunnel to the host router we get security from evesdroppers on the WiFi network, packet spoofing, and a very simple addition to our billing rules.

On the other hand the host router can still see the users packets, even if they are encrypted with HTTPS they will at least know what domains the user is visiting.

This IPv4 light client solution proposing is an improvement on the security properties of normal captive portals. But it's also a step backwards from the security promises that can be made by Althea routers.

We've decided to move forward with light clients and this security model for the time being.

Achieving the same privacy guarantees routers provide on Android would require a rewrite of WireGuard Android to produce nested tunnel packets natively. A task I estimate would take several months of work.

This way we can get a proof of concept out the door more quickly and easily and iterate from there.

In the developing world smart phones are the ruling platform. Home internet and by extension home routers as a delivery method for that internet hardly exist.

Althea maintains an incredibly low entry price for home routers, on the order of $20. We’ve worked hard to make this low entry price possible partly because we have a aspirations of introducing much more affordable home internet into some developing markets.

But it would still be great for people to be able to buy a connection directly via their phone. You can’t really compete on price with something people already own.

With this in mind we’ve started an Althea android app.

How it’s structured code wise is pretty interesting, we’ve pulled in the userspace encryption implementation from WireGuard android into our own app. This provides the encryption we need to connect to Althea exit servers since we just use a standard WireGuard tunnel internally.

This means that we don’t have to re-do all the hard work of creating a WireGuard for android client in the first place and can take advantage of the excellent engineering common to all official WireGuard implementations.

Next up we implement a stub client of the babel routing protocol I say stub because it is very much not a complete implementation and that’s intentional. This client has no support for advertising arbitrary routes or routing packets, this is intentional.

Phone WiFi systems are too weak to effectively provide internet service in a peer to peer fashion. Not to mention that Android is a pretty restrictive development environment when it comes to things like modifying WiFi and Bluetooth from inside an app. This is understandable due to the tracking possibilities tracking people by checking nearby WiFi networks is very common but unfortunate if you would like to have phones dynamically connect to each other without an existing network being shared.

By restricting the Althea phone client to ‘bandwidth buyer only’ we can get the engineering down to a manageable scope without too much trouble.

The source code can be found here and I encourage everyone to follow along with development. I plan to have a very rough proof of concept (hard coded user credentials) ready before Turkey day and then flesh out the UI and user experience during the first week or two of December.

Also expect an update tracking some of the difficulties of developing a networking application on Android, so far it’s been an experience full of pitfalls and unexpected behaviour.

What’s New in Beta 10 RC4

• Added toggle to ease scd -> mcd transition for the xdai bridge.
• Added Wyre integration
• Added Lagos exit
• Added the ability to remotely change release feeds of routers so that releases don't have to be global
• Compressed logging saves bandwidth when metrics collection is enabled, 95-99% improvement
• Allow the free tier to be 'router only' when users haven't paid depending on organizer configuration
• Change Oracle behaviour so that a user can refuse price oracle updates and still receive other settings updates

Beta 10 originally had ambitions of including IPv6 support, which is <languishing in this pull request> right now due to lack of time to properly test. What did end up making it in are features focused on improving the developing world experience in line with our Abuja deployment and a few quality of life updates for existing users.

Logging bandwidth isn’t really a problem in the developed world as it’s only significant for relays but for deployments with more restricted bandwidth you really need to make every byte count.

Likewise the default ‘free-tier’ of 1mbps is a practical solution in markets where streaming is popular but as we try and serve a more diverse group of people it’s hardly a one size fits all. This adds a toggle option for specific networks where free bandwidth isn’t a practical choice.

The other two major changes that made it into this release are to facilitate the change to Multi-collateral DAI and allow users in the United States to buy Ethereum using a credit card directly from the router. Wyre (the payment processor that allows this) is still in the trial phase with us right now but we’re optimistic that this will make things even more seamless.

The network in Tacoma is now live! Tacoma Cooperative Network, recently launched its network in the diverse, urban neighborhood of hilltop. Our community has faced a multitude of issues accessing reliable, affordable internet,”said Ann Dorn, organizer at Tacoma Cooperative Network.“We’ve talked to several people who live in areas where there’s only one service provider, which is either too expensive or too slow. We invite residents to join us as a member, not as a customer, so they can help shape the service to best serve their needs.”

“We’re passionate about building democracy and changing lives by making the internet affordable and accessible,” said Bee Ivey.“We have structured ourselves to be member-owned and run, so this is truly internet for the people, by the people.”

Having affordable access to the internet isn’t just a rural problem. One of the first connected people, Cheryl Smith, didn't have any internet service until recently when she got connected to the Althea network. Her and her sister are both disabled and live together in a small house in hilltop. Cheryl is going to school for a master's degree in Business Management, but they have no vehicle, so she rides a bus where she can or uses an older electric wheelchair.  To do homework, she had to go to the local library to use their computers and the internet - they just couldn't afford internet in their home.

Cheryl became one of TCN’s early supporters when Bee met Cheryl over the summer and told her about their cooperatively-owned internet network. With Althea you only pay for what you use, and the network is cheaper than other alternatives. Even if Cheryl has a hard month, she won't have to decide between feeding her, her sister, and her family of pets, or being able to comfortably do her homework and watching streaming video. With her new high-speed Althea internet, she can now access her health records online, keep in contact with family, and finish her education.

Our network in Nigeria is poised to launch! Backhaul will be installed any day now which will bring cheaper and faster internet to the residents of the capital city, Abuja. Local Althea organizer, Yakubu Yakubu or Yaxx, will be managing the deployment of the network there and earning a monthly subscription fee for every subscriber on the network. Local businesses and homeowners, like Yaxx’s cousin Usman, will put long-range wireless gear on their homes to resell bandwidth to neighbors.

We are also excited to announce that we will be partnering with MakerDAO to expand this network in Nigeria. People will be able to buy Dai directly on their home router dashboards to pay for service, or they could buy Dai in local stores with cash. “Althea’s networks are exactly the kind of real-world, empowering systems where we believe Dai can truly make a difference. We are excited to continue to build out bandwidth networks with Althea in places that deserve better infrastructure,” remarks Greg DiPrisco, head of business development at MakerDAO.

Another key development is our partnerships with middle mile carriers and local ISPs, who will allow local organizers to buy bandwidth on a per GB basis instead of a costly long-term lease. For our Newberg network, we will be partnering with the city of Sherwood to locate the gateway, and local organizer, Austin, can get started connecting people right away. In Grand Island, New York, local wireless ISP in nearby Buffalo, Rooftop Data, will be partnering to offer wireless backhaul to the Island. As our reach grows, we are seeing that many incumbents and municipalities are interested in these partnerships.

Althea networks bring faster, cheaper internet to communities and opportunities for ongoing revenue for organizers and community members. Now is a great time to start an Althea network! You can learn more about how things work and submit a proposal at www.republic.co/althea.

Althea organizer, Yakubu Yakubu (or Yaxx) resides in the capital city of Abuja. It’s here that he will be launching the first Althea network in Africa. “We’re in the dark here,” Yaxx says. “Local schools and businesses can’t get the access they need to compete in today’s global economy”. Yaxx hopes that setting up an Althea network will allow his neighbors to resell bandwidth at a more affordable rate and make money too. Once the initial wireless infrastructure is built, some of the participants may elect to sell their bandwidth from their wifi hotspot to nearby mobile devices. This will increase the reach of the network and revenue for relays.

Althea’s first node will be at an apartment complex near the heart of the city-owned by Yaxx’s cousin Usman. Small businesses and a park are nearby, with many middle-class homes and apartment buildings. Usman will set up a fiber connection to his building and place antennas on the roof. These antennas broadcast the internet to nearby homes and businesses. He’ll make about 10 cents for every GB used. Usman works at the airport and sells cars, and he’s excited about the extra income he’ll make from operating an Althea gateway.

Yaxx doesn’t make money from reselling bandwidth like his cousin Usman. Instead, he will act as an organizer for the network — helping people install antennas and being there if something goes wrong. He earns a monthly subscription fee for this work from every router on the network. He also helps coordinate and partner with other local people who want to become relays or gateways, like Usman.

This network runs on Dai, a stablecoin developed by MakerDAO. This is a type of cryptocurrency that always stays equal to the US dollar, eliminating the volatility that has kept cryptocurrencies like Bitcoin from being used in everyday commerce. People will be able to buy Dai directly on their home router dashboards to pay for service, or they could buy Dai in local stores with cash. “Althea’s networks are exactly the kind of real-world, empowering systems where we believe Dai can truly make a difference. We are excited to continue to build out bandwidth networks with Althea in places that deserve better infrastructure,” remarks Greg DiPrisco, head of business development at MakerDAO.

Big telecom monopolies dominate the internet landscape in Nigeria, keeping prices high and much of the population without access. Here, only 46 percent of the country are internet users and most of that usage is via cellular carriers, providing less than 1Mbps with prices around $3 a GB.

Having affordable, fixed internet access at home will transform people’s lives here in Abuja. Everyone in the family will have much better access to educational and economic opportunities. Businesses will be able to sell and ship products online. Hospitals and medical centers will have access to more knowledge and expertise. Using Dai to buy and sell bandwidth will quickly become the new normal here, and as the network expands in this internet vacuum, the new normal throughout Abuja, Lagos and all of Nigeria.

The self hiding log message

In our second most recent release Beta 8, there was a neat little line the changelog mentioning

Multihomed exits are now on by default, increasing reliability.

This is a feature I've been working on in the background for quite some time. Most modern web applications are hosted in clusters of servers so that any one can fail without taking the whole service down with it.

'Multihomed exits' describes moving our exit servers into this sort of architecture, where one or more than one could fail without significant impact on it's users.

This is actually quite technically challenging because of the encryption we are using. WireGuard provides Althea connections a property called perfect forward secrecy, this means the server uses it’s public encryption key to negotiate another encryption key that will be discarded by both parties at a later time.

Even if a hostile entity decides to capture traffic for years and then some day gets it’s hands on the exits encryption key it still won’t be able to decrypt the traffic captured years ago. Because the traffic is actually encrypted with a session key that was generated by both parties and then scrubbed from existence after exactly a minute.

While this is an excellent privacy feature we can’t just redirect traffic from one exit to the other, even if they share the same encryption keys, the session key will need to be securely re-negotiated on demand. A feature WireGuard does not provide.

Cloudflare ran into this same issue with their WireGuard based WARP VPN. Their solution is actually remarkably similar to ours, which I find somewhat disappointing as I was hopping to find a superior one.

While a single Cloudflare server failing won’t take down everyone on their WARP service but it will disrupt connectivity for all the users on that machine for a minute.

Cloudflare uses their own packet tagging system we re-direct traffic with Babel to achieve the same effect that we get with Babel redirecting traffic. The connection drops for 60 seconds while the user waits for WireGuard to re-negotiate the key and then the user is back online on the other exit.

Obviously that's a pretty disruptive wait but it sounds pretty easy to deal with, just make sure that traffic doesn’t get redirected unless the primary server is actually down.

As I learned the hard way that’s not actually so easy. Babel is designed to switch connections proactively as it tries to maintain a solid network out of a series of unreliable links.

I thought I could stay ahead of this by configuring Babel with a very strong preference for one exit server. I tried this solution and after some testing and the design of a monitoring tool I moved it into production.

Everything seemed perfect, even on prolonged observation. I congratulated myself on a job well done and moved on.

Over the next few weeks reports of momentary connectivity outages rolled in. But none of my monitoring was being tripped.

I was puzzled for a few weeks. The reports where very infrequent but I couldn't’ deny that something was going on that I couldn’t see.

Eventually I realized that I had designed my monitoring with a fatal flaw. I was using the connection that was being disrupted to monitor connection disruption. Very short disruptions that resolves themselves in under a minute would have their notification messages lost as the connection to the new exit server would never finish.

After rolling back the change problems abated, Cloudflare avoided solving this problem for good reason. It’s complicated and will require nothing short of shared cryptographic session state across a cluster of exits. A challenge it seems neither of us are really prepared to take on at the moment.

For the time being we will have to satisfy ourselves with scripted fail over to our secondary exits rather than the dynamic latency and packet loss detecting fail over that Babel would have allowed.

Nodes won’t accept new connections after months of uptime

As our test network grows larger the amount of node churn any individual node sees inevitably increases.

We started to run into another puzzling bug, after several thousand different node connections and reconnection an Althea router would be unable to successfully connect with new peers.

This condition would take weeks or months of uninterrupted uptime to appear on production routers, but appear it did. The root cause turned out to be rather interesting.

Babel (our mesh daemon) needs to be started using an interface. But since all of the interfaces we talk to peers over are virtual encrypted links created by Rita (our billing daemon) its sort of a chicken and egg problem. My ‘clever’ solution was to launch Babel bound to the systems loopback interface.

This turns out to trigger buggy behaviour in Babel where any interface going from an active to an inactive state will trigger a memory allocation in Babel.  It’s pretty similar to the bug outlined here. At first glance this sounds like the sort of memory leak that would eventually crash the program. Which would be fine as it would be restarted by our downtime watchdog script.

But due to a quirk of how exactly interface pointers are allocated only additional virtual memory is consumed. Presumably because malloc is run but the memory is never written to before the handler exits. So eventually Babel would be stuck unable to allocate additional memory from the operating system but otherwise continuing to function perfectly.

Our latest stable release, Beta 9 incorporates a fix for this problem. Ironically it turns out feeding babel the name of an interface that doesn’t exist is much safer and more gracefully handled.

Another lesson for the ‘too clever for my own good’ pile.

Development status

We’ve been working on stabilizing the Beta 9 release for the last couple of weeks and with Beta 9 RC5 we’re going to call it a wrap and move onto more ambitious features for Beta 10. We really wanted a stable base for our first Altheahoods as they get off the ground and Beta 9 will provide that.

At this point we have a pretty rock solid product where nodes can easily accumulate months of uptime without failures. The primary goal of all future development is to add new functionality without changing that.

Beta 10  will include compressed telemetry logging, which saves quite a bit of upload data for large nodes such as gateways which often have telemetry on. It will also include our first foray into IPv6 dualstacking for user devices. Which is something we’ve avoided so far for compatibility reasons despite our own internal network being IPv6 native.

As user counts grow we really don’t have any choice but to get dualstacking (providing user devices both an IPv4 and IPv6 default route simultaneously) figured out. Otherwise our exits will require an unsustainable number of IPv4 external IP addresses to map NAT traffic.

Beta 11 will be updating Babel to include upstream fixes and to add the ability to set per-interface prices. This is very important for relays that are both serving individual users and other relays. Otherwise you either have to provide all the direct users the wholesale rate or your wholesale buyer the user rate. Neither is sustainable as networks get deeper and less tree-shaped.

Physical network

These nodes are involved in actually connecting Althea users to the internet.

Subscribers (end users)

Althea’s goal is to provide faster, cheaper internet access to subscribers. A subscriber has an Althea router into which they have loaded tokens (a dollar-equivalent stablecoin). They’ll usually also have an antenna on the outside of their house, known as a CPE (customer premises equipment). This is the same kind of antenna used by centralized wireless ISPs. Some subscribers are connected over wires, for example if they are in an apartment building, or if they have run an outdoor ethernet cable from their neighbor.

Subscriber’s routers automatically pay for their service in two ways:

1. Bandwidth payments: This is a per gigabyte price, similar to how cell phone data is billed, but much cheaper. This is paid directly to the relay(s) to which the subscriber is connected. A subscriber’s router will choose whichever relay is offering the best combination of price and quality.
2. Subscription fee: This is a fixed monthly fee paid by automatically the user’s router to the organizer of their local Althea network. This compensates the organizer for providing technical support and service, and allows them to spend time and money promoting the network in their area. Organizers support the network as a part time effort or full time job.

Subscribers connect to the internet over exit nodes, which are just like VPN servers. A subscriber does not need to worry that the relay(s) they are getting internet access from can see their browsing history, and relays and gateways don’t have to worry about getting in trouble for anything subscribers are doing on the network. The exit node also helps keep relays honest by performing automatic speed tests behind the scenes.

Relays

Relays make up the physical infrastructure of Althea. They do the work of transmitting data across town and getting internet access from gateways to subscribers. Relays can come in many shapes and sizes. A few common examples:

• A smaller relay that might only be connected to one other subscriber using a second CPE, helping their neighbor get access while making a little money.
• A network of users who are both subscribers and relays, where every house has several links to other nearby houses. In this type of network Althea’s routing protocol balances data flows over all available links. Nodes switch from buying to selling bandwidth from second to second.
• A relay that uses sector antennas to connect to up to 50 subscribers. Sector antennas have a wider beam width and are designed to have a lot of CPEs connected to them.
• A relay that uses a faster point to point antenna to connect to another relay which is connected to a lot of subscribers.
• A relay that is in an apartment building, selling bandwidth to tenants over the building’s wiring.
• A relay that is selling bandwidth to a neighbor over an outdoor ethernet cable. These are common in the Althea network in Medellin.

There’s actually no difference between subscribers and relays in the Althea firmware. In a few of the above examples, relays are subscribers partially offsetting their own internet usage by selling bandwidth. In some of the other ones, the relays are professionally run.

Every relay in an Althea network earns a profit on top of what it buys bandwidth for. For example, if you are buying bandwidth from a neighbor for $0.10/gb, you might charge a profit of $0.05/gb and earn a total of $0.15/gb.

This means that subscribers will tend to get better prices the closer they are to a gateway, but we don’t expect prices across the network to be as variable as you might think, because relays will give each other discounts in order to be able to reach new subscribers. If I’m selling you bandwidth at $0.10/gb (same as I charge the 10 other subscribers connected to me), but you have the ability to reach 50 new customers, I might want to sell you bandwidth at only $0.02/gb and make twice the money I am now.

This market based system will also incentivize “bandwidth arbitrage”- when someone realizes that they can buy bandwidth in a cheaper part of the network, set up a long range link, and sell the bandwidth in a more expensive part of the network. Althea’s price-aware routing protocol allows routers to switch over to cheaper sources of bandwidth within seconds.

Gateways

Gateways physically connect an Althea network to the Internet. The technical definition of an Althea gateway is a node that sells bandwidth using Althea, but gets the bandwidth somewhere else, usually a monthly payment to an ISP. You may be wondering- where does the internet “come from”? The short answer is that there are wholesale ISPs - known as tier 1 or tier 2, which have a nationwide and global reach. They specialize in connecting a city or a region to the whole world. Retail ISPs, or tier 3, offer plans to homes and businesses while purchasing service from the wholesale ISPs.

Althea currently competes with the retail ISPs, since they are often worse run and more monopolistic than the wholesale market. A typical Althea network has one or more gateways buying internet wholesale and selling it into the network. Any number of nodes in an Althea network can be acting as gateways, and they compete against each other to provide bandwidth to the network.

People running gateway nodes do not have to worry about what people using their bandwidth are doing, and users don’t have to worry about gateway nodes seeing their traffic because of exit nodes.

Network Organizers

Althea network organizers help subscribers in their area install the equipment to get on the network, and provide technical support to subscribers. They also help promote their network to get subscribers to sign up so that relays and gateways can make money. In these roles they are much like traditional ISPs, except for the important difference that they do not own the infrastructure that the network is built on (this infrastructure is owned in a distributed manner by individual relay and gateway nodes). In many traditional ISPs, the role of owning and monetizing infrastructure dominates, and customer support often falls by the wayside.

Network organizers receive a fixed monthly fee from every router on the network, in contrast to relays and gateways, which are paid directly by the subscribers and other relays connected to them. An area, and the Althea nodes in it, can actually have several different network organizers competing with one another. Organizers run virtual networks called “subnets”, and any given relay can be members of several. This means that if an organizer is not providing good customer service, subscribers can easily switch to a different organizer while remaining connected to the same relay.

This flexibility also benefits the organizers. By spreading the infrastructure investment of the network out among property owners in their community, the network can grow larger faster, while organizers focus on customer service. By aligning incentives among many people in the area, the Althea network as a whole has an advantage over traditional ISPs.

Exit nodes

Exit nodes logically connect one or many Althea networks to the Internet. They often will be in another city at a data center, rather than being part of the physical network. They provide the public IP addresses used by Althea subscribers to access the public internet. They also keep subscriber’s browsing history secure from the physical nodes which make up the network, like a VPN. The Althea team runs default exit nodes, but it is also possible for a subscriber to use a different exit node.

Blockchain network

The blockchain processes transactions between nodes on the physical network. The currency used in transactions between routers is a stablecoin, which maintains its value at exactly 1 US dollar. This way users do not have to deal with currency fluctuations. There is another token on the network called ALTG, or Althea Governance token. This is used to govern the network as explained below.

Blockchain Validators

Validators compile individual transactions into blocks. In this way, they are similar to miners in Bitcoin. Bitcoin miners compete to mine the next block by trying to waste the most electricity. Althea validators compete by getting the most Althea token holders to delegate coins to them. When you delegate to a validator, it’s a vote of confidence that they will create blocks correctly and stay online. If they don’t, all token holders that have delegated to them will lose 5% of their delegated coins. Validators take a small percentage of the block rewards and transaction fees they earn, and pass the rest on to the token holders delegating to them.

Validators also play an important role in governance. Token holders can vote on proposals for upgrades to the blockchain’s code (and maybe someday physical node’s code). But if a token holder doesn’t care enough about an issue to vote, the validator they have delegated their tokens to can vote with those tokens. In this way, the governance system is like a representative democracy.

The Althea blockchain is built on the Cosmos blockchain framework, so you can learn more about how it works by studying Cosmos.

ALTG token holders

As explained above, these people have the ultimate control over the Althea blockchain by staking their coins with the validators who run the blockchain software. They also receive transaction fees from every transaction between routers. These tokens will be mostly held by the Althea community. As of 2019, we are granting them to people who are working on starting Althea networks. We may also grant some to important players in the open source internet world, such as standards bodies like the IETF and others. The company behind Althea (Hawk Networks, inc) will hold 25% of these tokens to finance ongoing development from the transaction fees.

We've come a long way in the last year we now have a much more varied network. Last year we had a pretty simple tree topology with no loops or any good way to take advantage of the redundancy and routing that babel really offered.

Now we're exploring that we can do with multihomed users and connections. Which brings us to what I was talking about in my last dev update.

As promised the automatic latency tuner I was working on is out and being tested in production. The diagram above shows a 10x improvement in latency spikes after deploying our new tool.

As far as I'm aware this is the first system of it's kind deployed for the last mile. How if functions is pretty straightforward and for the time being flawed but still effective.

Each router passively observes the status of it's neighbors connections by scraping Babel, when the round trip time to it's neighbors starts to spike it will reduce the maximum allowed throughput on that connection by 20% in order to fight bloat.

What this is really tuning is the shaper in the Cake traffic queue dicipline by reducing the maximum bandwidth on the link eventually Cake becomes the bottleneck and it can carefully prioritize traffic across the limited link instead of allowing the link to increase latency and disrupt user experience.

There are several future improvements to be made to the design of this system, mostly around reducing the rate of false positives by observing things like how much traffic is actually traversing the link at any given time and by working with unidirectional latency rather than round trip time.

But for the time being it's already providing a nice improvement for users.

Gateway-client corner case billing issues

The billing issue I talked about last week has been identified as a variation on the gateway-client corner case. An edge case that has plagued us for quite some time.

It comes up when a user wants to run Althea as a standalone device on existing backhaul by being both a gateway, a relay, and a client all at the same time. This is problematic to handle for a number of reasons, the gateway must complete somewhat conflicting goals of sending all user traffic over the encrypted exit connection while still relaying packets for other routers to the exit using the backhaul directly.

At some point during this forwarding process the gateway counts it's own client traffic as a billing credit and throws the whole system off. This is pretty easy to fix by simply not using gateways as clients somthing that's already universally true in our actual deployments.

Most people don't spend their time plugged directly into a server in the back room.

We just need to finish unravelling some other technical debt and we'll completely remove this corner case from v10 of the firmware.

What's new In Beta 9?

• Automated detection and resolution of latency spikes
• Support for Multiple LAN ports
• Support for WAN ports with static IPs
• Logging is now easily configurable from the dashboard
• Fixes for the /neighbors endpoint
• Dependency updates for Clarity and Rita
• Reduced image size by excluding trace level logging messages

Wholesale Backhaul

Buying backhaul is the basic “cost of goods sold” and determining both the cost and location will impact many of the future aspects of the network. The home, business, or tower where the incoming wholesale fiber connects to an Althea router and then broadcasts to the community is referred to as the “gateway” in an Althea network. This can be in a location which the organizer directly owns or rents, or it can be a partnership with a business, homeowner, or even a school or hospital. Middle mile and backhaul providers will need a physical address to quote with, so start by locating a few different prospective locations.

A few other ideas to consider when searching for backhaul:

• Check with your municipality. Where are they getting their connection? Can they lease you their fiber? Ask them about the carriers they are aware of.
• Check with nearby power companies or electrical co-ops. They sometimes have a fiber network.
• Get Wireless backhaul from the next town over's WISP. Use the FCC's broadband map to find out who is operating in your area.
• Find out who your competition is leasing fiber from.
• What are your anchor locations, like a school or health care center - can you partner with them to lease fiber?
• Get quotes from sales people who can provide multiple carrier quotes. We can help with this one if you provide addresses to quote from.

Competition

It’s also important to research the competitive ISP’s recurring and upfront costs, speed tiers, and customer service. In the United States, the FCC’s broadband map and www.broadbandnow.com will give you a starting point. Understanding the competitors will give you an idea of what speeds and prices you need to provide, and it may help you determine which messaging will work best in your area. Talking to local real estate agents might also provide perspective on the local competition. Keep in mind that even when people are unhappy with their current provider, the hassle of switching providers is a big barrier to changing their service.  Anger towards incumbent providers often only results in a one to three percent take rate, so you must also be a clearly better option in terms of service, cost, or performance.

Community

Understanding your community will involve both on-ground learning and research. Starting with demographics will give you an overall picture of age groups and income, but engaging with key community members is equally important. Community members or key groups can help answer questions like  - What can people in the area afford? Are you in a multi-lingual community? How might a new technology be perceived? Engaging with your community will also help you learn more about potential partners like universities, hospitals, or community centers who may have their own digital equity and digital inclusion projects.

From this research, you can begin to estimate take rates and target prices. Can you clearly position your Althea network to offer better prices, service, or performance? A compelling market fit is important to the start and early growth of the network. Once an Althea network is established, the relays and gateways are incentivized to promote and help your network grow organically.

We apply latency control judiciously throughout at every Althea node in the network, with the ultimate goal of fairly allocating bandwidth at any given bottleneck. This behaves very well so long as the bottleneck is one of the Althea nodes, but the intermediate radio links don’t behave nearly as well when overloaded.

Here’s a pretty classic example of an overloaded wireless radio introducing bufferbloat. The connection goes from a very excellent latency of 7ms to over 150ms as the wireless radio accepts more packets than it can send into it’s buffer before playing them back several hundred milliseconds later.

This is a pretty well known problem in the WISP industry. The usual solution is to over-provision backhaul links pretty significantly. There are alternatives that involve shaping traffic back at the datacenter, but those reduce capacity at the edge by deliberately reducing link over-subscription.

Both of these solutions are more manual than we would like and have costs we would prefer not to incur. We would prefer to spend that same money on extra links to simultaneously improve capacity and reliability.

So far the design of Althea has modelled antennas simply as wires. But situations like this and less subtle problems like outright bad radios will require some careful thinking and design. Ultimately our goal is to eliminate the hand tuning, over provisioning, and prolonged link investigations that WISPs often have to undertake.

Fortunately already have latency and packet loss estimations provided by Babel that should allow us to make educating tuning decisions automatically. As far as I know a system like this would be the first of it’s kind, so I’m excited to share a prototype with everyone hopefully in the next couple of weeks.

Billing testing and quirks

When I introduced our improved integration tests a few weeks ago it was with the intent of improving billing convergence, where all nodes agree on what every other node is owed.

Thanks in large part to that improved testing framework billing convergence has been flawless for more than a month now. This is great as lack of convergence is what causes the network to be disrupted due to billing disputes. But despite the fact that nodes all agree on what they are paying each other now there are some quirks in how billing is computed that result in the numbers being inaccurate compared to what a human would expect when configuring specific values.

For example if the exit sets a fixed profit margin of 1000wei/byte on top of the cost of returning bandwidth it will end up losing money.

Intuitively this shouldn’t be possible, after all the exit is adding it’s margin on top of whatever the actual route price is. But the way we compute fees actually plays a role here. Since all nodes will spend 5% of their total payments on fees the exit charging less than 5% of the total path price ends up losing money. Other nodes are also affected by this but not nearly to the same degree.

This isn’t exactly a problem of correctness, as our prices are being expressed in exactly the way we described, but it’s definitely not what someone would expect.  

The proper way to solve this is to change how price setting is described by the user such that the configured margin will always bake in transaction costs. I’m hoping to get a patch for this up in the next couple of days.

Over time we’ve moved from billing never working to billing being perfectly stable, hopefully this will be the last hurdle before billing is not only stable but also always works exactly as expected.

The proposal passed last week, and we are currently planning how to do it.

We will most likely use a script that will wrap the gaiad binary. This script, when run, will start gaiad and also start a process that polls for the current block height. When a predetermined block to fork on is reached, it will swap out the gaiad executable for a new one, stop the chain, modify the genesis file appropriately, and start the new executable. If the script is started after the changeover, it will do these steps if necessary, or otherwise just start running the new executable. This way, it should be usable as a drop in replacement for gaiad with systemd or any other system management tools.

Once these details are finalized, we'll make another governance proposal with the exact forking height, modifications to genesis.json, and the recommended forking script described above.

The headline feature for Beta 8 is the automated Xdai bridge. Which will allow us to provide one of the most demanded features without compromising on liquidity.

A ‘stablecoin’ is a token that attempts to have a fixed value relative to some peg, usually the US dollar. Allowing the user the advantages of transacting using cryptocurrency without the pitfalls of rapidly changing exchange rates.

There are plenty of stablecoins on the Ethereum blockchain, but you can’t generate a transaction of any kind without paying fees in ETH.

If a user deposits $50 in the DAI stablecoin into their router it's completely useless. Any transaction attempting to move it will need ETH to pay the fee.

We got around this by adding support for the Xdai blockchain, which is a side-chain where transactions can be paid for in the DAI stable coin. This also lets us get away from paying the Ethereum blockchain’s often slow transactions, roller-coaster gas prices, and lack of finality.

It did come with a catch though, users can easily aquire ETH using Coinbase or any one of many other exchanges. Buying DAI is slightly more difficult, and getting XDAI is a pretty technical process. For our main test deployment removing the easy monetary on-ramp provided by Coinbase was too big of a loss.

So what we’ve done in this release is design a fully automated a bridge, the user merely has to deposit ETH and the router takes care of the rest. The money will be exchanged for DAI, then transferred across the cross-blockchain bridge to become Xdai.

When the user wants to withdraw money the funds are brought back to the Ethereum blockchain and deposited into the users wallet as Ethereum for maximum liquidity.

You’ll notice that this is a pretty direct mirror of the setup we propose for the Althea blockchain. We intend to use Xdai as a stopgap and testing ground before moving to our own chain.  

Xdai is an Ethereum based proof of authority chain. Whereas the Althea chain will be a Cosmos based distributed proof of stake system.

Proof of authority means that a small set of validators (currently 9 for Xdai) are the definitive validators for the blockchain. This does use a lot less power than mining, but is more centralized. Even the Althea testnet already has more than 20 validators and more are free to join at any time without lengthy vetting and manual addition.

The production Althea blockchain will have a group of at least 100 validators which anyone can join. We’re hoping to be able to expand that to 300 validators.

Cosmos should also be (at least in theory) faster and more flexible in terms of on chain contracts. But mostly we care about the more flexible and open proof of stake system.

What’s New in Beta 8

• Improved and stable Xdai bridge, with graphical cues and much better error handling.
• Updated dashboard with better setup flow and checklist for items to complete.
• Oracle has improved flexibility so that we can easily move existing deployments onto a subnet dao.
• Fixes for Rita crashing when a password is set
• Multi-homed exits are now on by default, increasing reliability.
• Routers now implement a debt limit to better support running with no balance for long periods
• More secure exit signup using libsodium

Here are the changes that were made to the chain's settings and state:

• Reduce governance proposal deposit to 1 ALTG. This lets us do governance proposals easily (it was 512 ALTG before, and most people only have 100 or 200 ALTG).
• Reduce unbonding period to 1 day (86400000000000ns). This makes it easier to take your tokens out of delegation to send them around and stuff. It was formerly 21 days.
• Minted 100 new ALTG for anyone who signed up for our validator notification mailing list at https://airtable.com/shrpjZDQYQa3Xd5oe. The newly minted tokens are to give people who weren't in altheatest3 a chance to get some tokens to start validating on altheatest4, and to give the old-timers a little extra worthless testnet pocket change.

Hard forks on Comsos chains are currently a surprisingly manual process. Here are the steps:

1. Everyone waits until the predetermined time.
2. Everyone stops gaiad, and exports a new genesis file which will be used to start the new chain.
3. Everyone makes any necessary changes to the new genesis file (at least updating the chain id), and replaces the old genesis file with the new one.
4. Everyone does gaiad unsafe-reset-all, and gaiad start.
5. Once validators controlling 2/3rds of the voting power on the chain have completed these steps, the chain starts again.

We were stuck between steps 4 and 5 for almost 24 hours due to several factors.

• First of all, a lot of people had issues with the error ERROR: error exporting state: failed to load Store: wanted to load target 206400 but only found up to 200000  This was caused by the state pruning algorithm. In the future, we will set the forking block in advance, and as soon as it is decided, everyone will restart their nodes with gaiad start --halt_height [fork_height]. This will make sure that the pruning algorithm is ready for an export at that height.  
• Because of these issues, some validators had to start using an altheatest4 genesis file from someone else instead of exporting and processing their own. This led to some errors, such as people mistakenly running the script on the already processed altheatest4 genesis and ending up with the wrong number of tokens in accounts.  
• Also, about 1/4 of the validators did not come online until later Tuesday. With the MIA validators, and the validators with genesis file issues, we were slightly under 2/3rds of the validators for a while. It also took a little while for the chain to start up once we got enough validation power online because it had been stalled so long (I think it's an exponential backoff thing).

Next validator challenge

I think we should do a hard fork to gaia v2.0.0, which I believe is ready. Here is the Cosmos Hub proposal about it: https://ipfs.io/ipfs/QmbXnLfx9iSDH1rVSkW5zYC8ErRZHUK4qUPfaGs4ZdHdc7. Additionally, we should do it as a scripted hard fork, which should minimize downtime, and be easier for validators. Validator Blitmore has already created a script which the Althea team can audit and possibly improve.

I've created a governance proposal to this effect: https://althea.zone/#/governance/2

I mistakenly titled it "Scripted hard fork to gaia v0.36.0". This is somewhat incorrect, since it is gaia v2.0.0, built with Cosmos-sdk v0.36.0. Now that it's on the blockchain it's permanent. Forgive my stupidity.

Most of my dev updates in the 40-60 range focus on stabilizing the routing software and we're getting into the tail end of that same process with payments. Many of the fixes in our Beta 7 release focus on payment bugs that affect as little as half a percent of all payments.

We're down to about one payment instability event that requires human intervention a week across the whole network. I think we'll be seeing one a month this coming month.

After more than a year of this stabilization treatment Althea's network stack is robust enough that networking failures due to the point to point radio software are more common than Althea device failure.

For want of a single transaction

How two http call timeouts took down our test network for about 30 minutes.

You can see a graph of payments between the gateway and the exit here, with the debt in Wei graphed over time. You'll notice how the paymetns become more frequent as we move into internet prime time and more traffic is being moved and paid for.

Around midnight PST a confluence of high traffic activities where maxing out our backhaul connection. This wouldn't normally be a problem except for a confluence of failures.

Transaction 0x744c7b7b... was part of a flurry of more than 3 transactions to keep up with the traffic spike. The gateway was successfully notified of the payment, but when it went to check if the payment was valid the request to our Ethereum fullnode timed out.

This normally isn't a problem, we load balance across two providers at random.

Except a minute later when the gateway tried to validate that same payment again it chose the same failed node at random.

It wouldn't get a third chance.

Without that payment there was a very short window where the gateawy determiend the exit had exceeded it's grace payment amount. The gateway began enforcement action by reducing the exit's bandwidth.

This set off a chain reaction as a fully saturated backhaul connection was limited to a mere 1mbps. The gateway had accidentally made itself unable to check any transactions and thus made recovery impossible.

Manual intervention restored functionaltiy within 30 minutes.

In response to this incident we started running the payment validation process every 5 seconds rather than every minute.

As a longer term solution this problem is part of the gateway client corner case. Where users expect a gateawy to also act as a client and it causes no end of special case requirements. We will probably move to doing things like verifying payments over the backhaul directly rather than via an exit.

Headline feature in Beta 8, stablecoin payments

We've finally merged the token bridge module. What this does is that it takes Ethereum deposited into the router and converts it to the DAI stable coin using Uniswap before passing it over to the Xdai sidechain.

This lets users take advantage of the relative ease of buying Ethereum without having to expose themsevles to price changes while using their Althea device.

It also provides insulation from gas price changes and other network disruptions that may happen on Ethereum.  

Transacting in native Ethereum will of course remain an option on the routers.

You may recognize that this is very similar to what we've been talking about with the Althea blockchain, the main difference is that the Xdai chain is 'proof of authority' meaning you must be approved to become a validator/miner.

While this is somewhat decentralized it's not as open as we would like for a long term solution. Our chain will be using a more open system where anyone can join as a validator.

What's new in Beta 7

• Upgraded to OpenWrt v18.06.4
• Experimental Xdai bridge support, routers in xdai mode will bridge over ETH automagically when deposited
• DAO address based pricing oracle more closely reflects eventual functionality
• Fixes to rare payment failures
• Fixes for importing backed up private keys
• Fixes for exits behaving improperly when private keys where imported
• Fixes for DAO payments sometimes causing the router to crash
• Long term memory for router debts to prevent restarts resulting in not paying other people
• Fix for the dashboard thinking routers had a password set when they did not
• Added a version toggle that allows devices to opt into early access updates
• Users can now visit althearouter.net instead of 192.168.10.1 to access their own router dashboard.

Earlier this week, we launched the first public testnet of the Althea blockchain, known as altheatest3. We had some tense moments when it wouldn't start, but in the end we got everything up smoothly. You can see our block explorer and validator list here: https://althea.zone/#/staking/validators.

It's quite decentralized- there are 19 separate validators, and no validator controls more than 5.97% of the voting power. 4 of those validators are run by Althea team members, but altogether, they only control 19.4% of the voting power. All of them could go offline without affecting the system, and they may.

It's now time to start stress-testing the system. We're going to be organizing a series of semi-weekly challenges for the validators. They are going to start out fun and cooperative, but will get increasingly challenging as time goes on. At some point, Justin is going to try to take down the blockchain using only a single blue wifi router.

We got feedback that people liked the cadence we established when spinning up the altheatest3: announcement, full instructions a week later, then it's go time at the beginning of the next week.

For our first challenge, we wanted to start easy. Maybe a few light governance proposals, etc. But there's a hitch in the plan- when we made the genesis file for altheatest3, we kept the Cosmos default that people needed to deposit at least 512 tokens. However, altheatest3 has only about 2000 tokens total. So it's going to be really hard to make governance proposals.

So our first event is going to have to be a hard fork, to altheatest4. This will be the same process as when the Cosmos hub forked to enable transfers. altheatest4 will have a few changes:

• Reduce governance proposal deposit to 1 ALTG. This will let us do governance proposals.
• Reduce unbonding period to 1 day. This will make it easier to take your tokens out of delegation to send them around and stuff.
• Mint a bunch of new testnet tokens for anyone who signs up for our validator notifications email list! The sign up form for the email list is here: https://airtable.com/shrpjZDQYQa3Xd5oe. The newly minted tokens are to give people who weren't in altheatest3 a chance to get some tokens to start validating on altheatest4, and to give the old-timers a little extra worthless testnet pocket change.

We'll be releasing full instructions on how to do the hard fork on Wednesday, August 14th. On Monday, August 19th at 3pm PDT, we will hopefully be switching the chain over.

You can now easily download and run a local Althea testnet through a series of highly instrumented tests with a single command.

Running the tests in one easy step

First have docker installed and running. Then run these commands. Word of warning this will blow away your systems docker cache so don't run it on a production server, the builds are pretty big and will fill up your root dir very quickly if we don't, you can edit test.sh if you want to avoid this.

Dependencies are Docker and a local WireGuard kernel module. This is mostly easily done on Linux. If you don't have a Linux machine setup you can see the test output on Travis.

git clone https://github.com/althea-net/althea_rs
cd althea_rs
bash scripts/test.sh

That's it! The entire test suite will run from start to finish in this order.

1. Unit tests
2. Creating a docker container and building Rita our biling daemon and Babel the routing protocol from scratch
3. Creating a virtual network using net-namespaces to connect 7 different virtual nodes.
4. Checking connectivity and optimal price/paths for all nodes
5. Registering client nodes with the exit and checking the various http endpoints for the router dashboard
6. Running a series of iperf tests to generate traffic and then ensure that the resulting billing output is consistent

This is a cool way to see everything at work and even mess around with it without having to get specialized hardware or do a lot of setup. The ~12 minute runtime isn't exactly ideal but it's a lot less work than any other form of of in depth testing that we have had before.

You could pretty easily generalize this test to more than the 7 nodes we test now. But I'll leave that as an exercise for the reader.

What's new in Beta 7

• Fixes for importing backed up private keys
• Fixes for exits behaving improperly when private keys where imported
• Fixes for DAO payments sometimes causing the router to crash
• Long term memory for router debts to prevent restarts resulting in not paying other people
• Fix for the dashboard thinking routers had a password set when they did not
• Added a version toggle that allows devices to opt into early access updates
• Users can now visit althearouter.net instead of 192.168.10.1 to access their own router dashboard.

This is a pretty quick release hot on the heels of Beta 6, but we have a lot of fixes to roll out so I figure we'll push it out quickly.

• The Althea blockchain will allow subscribers to pay for internet access in a dollar equivalent token (stablecoin) rather than needing Ethereum for transaction fees
• The Althea blockchain will be proof of stake, and will consume as much power as a moderately sized website.
• There will be Althea Governance Tokens (ALTG). Subscribers will not need to buy these except if they wish to act as a chain validator and participate in the proof of stake system
• The transaction fees from subscriber’s routers exchanging stablecoins will be paid out to the validators and used to help support further development of Althea

Up to now, we've been running our networks on the Ethereum blockchain. The reality is that Ethereum simply isn't suitable for our use case. Ethereum's great for manual, medium or high value, sophisticated transactions, such as MakerDAO and Uniswap, but it has some issues for small automated payments such as those in Althea. At least once a week, some automated arbitrage script or token sale on the Ethereum network pumps the average gas price way up, putting router payments on hold. This destabilizes physical Althea networks which are providing the essential service of internet access.

We're doing this with Cosmos, which is a framework for making blockchains. It's proof of stake, which means it doesn't waste electricity like Bitcoin or Ethereum. Instead, the blockchain is kept secure by "staking". There is a special type of token on the blockchain that can be staked by delegating it to validators. When someone has tokens staked, they get a small percentage of the transaction fees on the network. The flip side of this is that if the validator you have staked to misbehaves, you'll lose a portion (currently 5%) of your tokens.

The staking token on Althea's blockchain will be known as "ALTG", or "Althea governance token". It's important to be clear that payments between routers will not be denominated in this token. They will be done in a stablecoin equal to the dollar. This means that users paying for internet will have an experience comparable to PayPal, Venmo, or any other electronic payment medium. ALTG is only for staking, and voting on upgrades to the blockchain software through the governance process.

You may be wondering who will get ALTG tokens. Some will go to the founders and early developers of the Althea software, some will go to investors, and we will grant 1/3 of the tokens to Althea network organizers and users over the next few years.

By using proof of stake instead of proof of work, we take transaction fees that on Ethereum would have gone to pay miners to waste electricity, and instead use that money to support development of Althea and to give control to users and organizers of Althea.

Ethereum 2.0 will also introduce a proof of stake system, which is much less wasteful. But in this case the transaction fees will still be going to early investors and traders of Ethereum. By using the Althea blockchain, these fees go to support development of the protocol. It’s important to note that subscribers will still use Ethereum to buy their stablecoins using MakerDAO and Uniswap when reloading their routers.

We are currently testing the blockchain through a series of testnets, and looking for validators. For now, there will be 100 validators on the chain. These will be the 100 validators with the most ALTG delegated to them. This set will vary over time as ALTG holders decide which validators they should delegate to. Running one requires extensive experience in devops. Mistakes as trivial-seeming as running both your main and your backup server at the same time can result in slashing and the loss of tokens.

If you want to become a validator on the Althea blockchain, read this.

Questions about all of this? Join our community call and ask away! https://airtable.com/shrCyyivU2gpm0AI6

We're currently in the process of spinning up the Althea blockchain. We are using Cosmos, a proof of stake blockchain framework. This post is for those who would like to help validate. A validator is sort of like a miner in a proof of stake blockchain. They don't need any special hardware or a large amount of power. Instead validators risk 'staked' tokens. A validator will lock up staking tokens and lose 5% of them if they misbehave.

Misbehaving is defined as:

• Signing two blocks at the same block height (forking the chain)
• Being offline (not signing any blocks) for more than 16 hours without first removing yourself from the validators list
• Losing your private key
• Other conditions that may compromise the security of the chain

Most people will not be validating themselves but “delegating” tokens to a validator. When you delegate your tokens you are adding them to that validators stake and you will split the profits from using your tokens with the validator.

What you will need to validate is:

• A dedicated server a decent amount of processing power and storage
• A ledger hardware wallet (optional)

Generally you want a server with at least 8gb of ram and 500gb disk. SSD storage isn’t as important as it is with Ethereum.

You can use a VM from AWS or some other hosting provider but recognize that there are extra risks to doing so. First off, the hosting company will have your private key, second if the hosting company spins up two versions of your vm you will validate two blocks at one height. This is a slashing condition. If you plan to stake an amount of tokens you care about, or are providing your validating services to others, I would suggest renting a physical server. For any serious amount of money rack your own hardware at a colocation center.

Racking your own machine has a side benefit of giving you physical access. This is where the Ledger can come in. Very serious validators use a ledger to hold their private key securely while they validate. Even if their server gets hacked the key will remain secure in the ledger storage. They will probably still be slashed (lose 5%) though.

There are many many colocation and dedicated hosting providers. If you can’t identify a good host in your budget through your own research, being a validator may not be for you.

Here are some good additional resources:

• Cosmos validator FAQ
• Certus One validator guide

How exactly do I validate on Althea?

As you read the following instructions, please do not hesitate to reach out in our Discord channel, in the #althea-validators room. We are aware that these instructions might not be perfect or might be hard to follow. We want to help you get set up.

Unlike a lot of other networks, Althea's blockchain is really one of the simplest parts of our infrastructure. Our routers do all kinds of complicated things to create autonomous local bandwidth markets, but almost none of this is on-chain. Our blockchain only needs to process transactions. Because of this, we are a very close fork of gaia, the Cosmos hub blockchain node software.

In fact, it's not even a fork right now. The only difference between the Cosmos hub and the Althea Zone is the genesis.json file, which defines the blockchain and it’s parameters.

Here are the instructions to join our current testnet: https://github.com/althea-net/althea-zone

Community Outreach usually starts by establishing a social media presence.  If you are planning a network primarily comprised of businesses, you may considering forgoing a social medial platform in lieu of a traditional website, and more direct cold calls. For this approach, it may be helpful to compile a list of nearby businesses and then send emails, call, or arrange meetings.

Facebook still is a good choice for most communities, because of it’s wide reach across broad demographics and personal approach. Facebook also has community groups, which is a great place to share your network or comment on the ubiquitous “What’s the best internet service?” post . Start by establishing your facebook page with a few introductory posts like a “get to know us” or information about the other existing Althea networks. Leverage your own social network to establish a foundation of support, even if your invited friends aren’t in the immediate area. As the project grows in scope, you can post updates, some information about how Althea works, and even some fun memes. Depending upon your community, twitter, instagram and nextdoor can also be useful.

Another important part of outreach is establishing a presence in the community. This can be accomplished by speaking at community events, becoming part of the chamber of commerce, or forming a partnership with an established non-profit or business. It is often said that people need to be exposed to something eight times before they are willing to commit to purchase, so it’s best to be involved in many different outreach opportunities. These partners can often help with the logistics of the network, or become your relays and gateways as well. Call or email a potential partner to set up a meeting, and then share your Althea project with them along with clear ways that you can work together.

Participating in fairs and other community events is also a great way to establish a presence and get people signed up to become a relay or subscriber. Many communities will post a calendar of events, which is a good place to find opportunities. You can also hold your own event, like a picnic or BBQ, or, in the winter, an informational event at a library. Keep your event fun or provide another benefit like a class on wireless security, so people will have a reason to attend.

The next step is to get the word out in the community. This is a direct outreach phase, and old fashioned door-to-door canvassing is often the best way to meet people. Canvassing can be intimidating, so practice before hand and follow these best practices:

• Make a goal and plan - How large of an area do you plan to cover? What’s the goal for number of sign-ups?
• Bring a buddy - Friends provide support and make the job easier.
• Write a script - We have some samples available if you are looking for ideas.
• Bring a clipboard - You may want to sign people up with a tablet, but old fashioned clip boards are also great. Make sure to fill in the first few entries.
• Bring materials - Have a flyer, door hanger, or other materials to leave with people you talked with.
• Follow up - Make a plan to follow up with interested people. Use a CRM or spreadsheet, like Airtable, to keep contacts organized.

Support your canvassing and outreach efforts with traditional marketing efforts, like boosted facebook posts, flyers, and yard signs. This adds credibility and boosts name recognition. Boosted posts can include updates and short videos of canvassing and outreach, or boost a link to your sign up page directly.

This is also a good time to send out a press release or get in touch with publications to let them know about your Althea project. Don’t overlook niche publications like senior, business, or university publications or a local radio station. Often your chamber membership will help you contact the right press people, and they can also help promote your project within their own business network.

A successful community outreach project will require time, organization and a plan. We realize this often can be the most challenging part of starting a network, and we are here to help! Using our Altheahoods program, we guide you through each step of the process and support you with marketing materials, PR, encouragement, and ideas.  The Altheahood in hilltop is a great example of what a good community outreach project looks like and with just a few weeks under their belt they are almost half way to their goal of 100 sign ups!

Sign up to become an organizer at www.althea.net.

We also launched a new Altheahood last month in the Napa valley of region of California. Internet access in this rural area is expensive and very slow, well below broadband speeds. While there isn’t any fiber available to the consumer here, there is wholesale fiber connections which makes this a great area to start an Althea network. We recently published a blog post which gives some insight about how the economics of these networks can look.

This month, Jehan Tremback visited Berlin for Interchain Conversations, a Cosmos un-conference. Over the next few months we will be building the Althea token (ALGT) using the proof of stake Cosmos platform. Althea Governance Tokens will not be used to pay for bandwidth or monthly subscription fee, instead subscribers will still use ethereum, and soon dai, to pay for their bandwidth. Validators and those delegating Althea governance tokens to them will earn transaction fees from each payment. We talked about Cosmos, links in Clatskanie, and chatted with hilltop organizer’s Bee and Ann about their deployment in our last live community call, which is available on youtube in case you missed it.

Justin Kilpatrick has been busy working out increasingly nuanced software bugs and rolling out the latest Beta 6 release, which will include a simple “neighbors” tab showing connected users.  The previous release included the highly requested bandwidth payment tracking to the dashboard where subscribers can view their usage and payments. Justin also recently gave a presentation at the NANOG conference and shared a fun video about Althea performance with CoDel.

The Althea founder team will be in San Francisco for a meetup in July -  details to follow soon. If you are ready to start an Althea network in your community or want to learn more about what that entails, visit our updated getting started guide or sign up to be an organizer at www.althea.net.

// removed and placed into error handling in payment controller
// time will tell if that was a good idea
// update_nonce(our_address, &web3);

Lesson of the week, if your gut tells you that something might be a bad idea you should probably listen to it.

The ‘nonce’ is the sequence number of a transaction in Ethereum, they must be issued strictly in order. If you have a published transaction with nonce 1 your next transaction must contain nonce 2.

I wrote this code on May 1st, the goal was to start managing our transaction nonces locally. Previously we had been able to get away with just asking the full node what our current nonce was and using that.

But if the blockchain was congested this was a problem. The full node will happily tell you the current valid nonce, but not how many transactions you have pending.

My goal was to let Althea devices manage multiple pending transactions at the same time. If we just incremented it locally this should work perfectly. In the case that something actually did go wrong we could just update our nonce from the full node at that time.

As I slapped this solution together I got a bad feeling. What if the full node lied to us? I really couldn’t think of a situation in which it would but I just couldn’t shake it.

It's around midnight on Friday, May 24th. Nearly a month after we put this code into production. My brother is in town for the first time a year and we're out getting drinks and regaling each other with the various ridiculous failures of our respective tech jobs when I get a call.
Over on the west coast its 9pm and prime time for internet usage in our test network.

Unfortunately over in the rainy cascades, all is not well. The relays have not been paid by the gateway in over a day and  high weekend usage had pushed them over the edge to demanding the late payment they where due by slowing down their connection to the gateway.

This inevitability caused the network to catch fire and my phone to ring.

It took me about 30 minutes to piece together what had happened. My gut feeling had been correct, there was a way for the full nodes to lie to us.

Remember what I said about transactions having to be in exact sequence to be valid? This was the key cause of our error.

Nonce 728 was submitted on May 23rd at 5am and made it into the blockchain a few minutes later.

Transaction 729 was submitted shortly thereafter but after hours of waiting it never got into a block. Here’s where everything goes horribly wrong.

When a transaction so broadcast into the network of Ethereum full nodes it’s in the hope that a miner (who creates blocks) will pick it up and place it into the blockchain. Exactly how long any given miner keeps transactions around waiting to put them into a block varies, but usually isn’t longer than an hour.

Our full nodes on the other hand, are not miners. They merely keep track of blockchain data and respond to questions from Althea devices. They had no configured timeout for transactions.

So when we submitted transaction 730 our full nodes thought this was perfectly fine, after all whenever 729 got in, 730 would follow shortly thereafter.

The miners, having discarded 729 after an hour or two didn’t agree. 730 was to them a totally invalid transaction, they didn’t have a 729 and without some way to fill in the sequence there was no way to publish 730.

On and on this goes, up to nonce 1000+ until my phone rings at the bar.

In the end my late night fix simply involved manually setting the nonce to the correct value and restarting the gateway, which immediately produced a new transaction 729 which is in the blockchain to this day. This is a classic sort of distributed systems bug, correctness is a matter of perspective.

Development update

Since our last dev update we’ve been working to finish stabilizing payments and prepare our own infrastructure (specifically the Rita exit code) to scale as our growth rate increases.

Some of these performance improvements have trickled down into the routers, which didn’t really need them so much but hey faster is faster.

We’ve also continued to patch issues, as the payment code becomes more and more stable smaller and smaller problems become apparent and are able to be tweaked. At this point we can maintain payment consensus for several weeks on end despite whatever chaos is going on in production. 4x sweeps in the gas price or antenna degradation and packet loss. We’re well into the long tail and I would expect within an update or two we may just have nothing to patch about billing anymore.

What's new in Beta 6?

---

• Password protection support for the router dashboard!
• New relay management screen to manage neighbours connections and payments
• Dramatic efficiency improvements thanks to async Babel interactions
• Faster saving for bandwidth use display features, now every 4 hours rather than 24 so less data is lost on reboot
• Use 105% of the median gas price in order to avoid the situation described in Dev update #70
• Devices will now apply spare change from slight overpayments before instead of after enforcing
• Fixed a bug where the exit may be paid twice in quick succession
• Fixed a bug where the babel routing table could grow to large for Rita to read

The Altheahoods program is going pretty well and all told. At this point it seems that getting backhaul on location will probably take longer than getting all the signups. Which is either really good or a scathing criticism of backhaul fiber providers depending on your perspective.

You will also notice that we’ve changed our domain (yet again) to althea.net, this was the result of a lot of thinking about our branding choices. We’ve needed to ‘drop the mesh’ for a while now as it tends to cause confusion in different audiences. It’s much better for us to define what we do rather than be pigeon-holed into what people think when they hear mesh (which can vary widely from venue to venue).

The bird also got dropped, since it didn’t match the new art and branding style of the website. Lets all have a moment of silence for flappy.

The tech

What is this technology called? Is it 5G? No. That's just how good radio engineering is these days. All of these capabilities are available in inexpensive, commonly available radio equipment today. It's in newer (802.11ac) home wifi hotspots, and it's in the outdoor antennas used by wireless ISPs to supply internet to millions of happy customers. Nobody bothers to hype it up or call it by a special name. It's business as usual.

Unfortunately, cell phone companies have not been able to take full advantage of these newer radio technologies. They have been hobbled by the aging 4G protocols, which still contain some of the DNA of old-school landline telephonics.

On a technical level, the 5G protocols are simply an initiative to bring cellular technology up to the standards of modern wireless internet, without the sub 30 megabit speeds and 70 milliseconds of extra latency that dog most 4G deployments. This speedup comes from the use of higher radio frequencies, or “millimeter wave”.

It's a reality of physics that while higher frequencies have a higher data capacity, they are not able to penetrate obstacles. You will not get the promised high speeds of 5G unless there is a clear line of sight between you and the tower. If you walk around the corner, or under a tree, you will be back on the same frequencies and same speeds as 4G.

The hype

“This network [5g] will usher in what I think will be the fourth industrial revolution” -Lowell McAdam, former Verizon CEO

“I see a huge quantum leap when going from 4G to 5G” -Hans Vestberg, current Verizon CEO

Why so much hype? The simple answer is that people love to buy stuff that has a higher number on it than last year. “5G” gives carriers and cell phone makers a convenient label to hang their marketing on. Radio speeds will continue to improve, no matter the label, but someone can sell you the new 5G phone tomorrow.

5G is also a potent lobbying tool. Politicians have never been known for their firm grasp of technical subjects, and the idea of “China winning the 5G race” has allowed the cell phone companies to whip our politicians into a frenzy. For example: T-Mobile and Sprint are currently trying to get the feds to make an exception to centuries-old antitrust law because of 5G and China.

The reality

The reality is that 5G is not about the 5G protocol. 5G is a coordinated marketing and lobbying push by the cell phone carriers. The benefits of 5G are enabled by basic improvements in radio technology, which will only be brought into the real world with massive investment, whether or not the 5G protocol happens to be keeping the bits straight.

That's not to say that 5G isn't a technological achievement. Letting cell phones seamlessly switch to high bandwidth millimeter waves (when they're not blocked by a window, a tree, or a passing bird) is great. The new millimeter wave towers will really shine in settings like sports and music events- where there's a high demand for bandwidth, and everyone is able to see a tower mounted over the stadium.

Given that the 5G protocol doesn't provide capacity benefits by itself, a bet on 5G is really just a bet on cell carrier's ability to allocate capital. For example, Verizon's push for home 5G is looking exactly like how a traditional wireless ISP hooks up its customers.

It's great that the cell carriers are entering markets like IoT and home broadband that have been hampered by lack of competition for so long. But a 5G monoculture would be no better than what we have now. Our broadband future will be brightest with a healthy mix of competition. Cell carriers with 5G, wireless ISPs, community mesh networks running long range WiFi, and even the bad old cable ISP monopolies all have a role to play. More choices and more competition equals better outcomes for consumers.

Small Business

An Althea network could be administrated by a small business as the organizer. Althea organizers help support the network with installation and customer support and can collect a monthly “subscriber fee” for each end user. This role fits well within the the structure of a small business who may have a manager and even a few employees as it scales up. A business makes it easier to take in revenue, maintain insurance, and manage expenses and taxes.  

One of the more challenging aspects of starting the network is finding backhaul (the fiber connection to the internet) and setting up a gateway node. For this reason, especially during the bootstrapping phase, this business may also elect to own and operate one of these gateway nodes. This gateway node collects fees for bandwidth on a per GB basis, often around 15-30 cents per GB. Because usage varies, the ability of the business owner to utilize both the bandwidth fees and the monthly subscriber fee can be very helpful.

Our best usage data comes from our network in Clatskanie that demonstrates an average of 180GB of usage per month, per household. This data closely aligns with the national average of 201GB in 2017, but is below what a recent OpenVault report indicates is an average of 268GB in 2018. If the gateway sets a price of $0.20 per GB they could expect monthly revenue of $36/per user/month. The business could then add to this revenue with a $10 monthly fee for an average revenue (given Clatskanie’s average) of $46/month.

The major operating cost of the gateway is the backhaul itself, which is often between $1000-$2000 for a 1Gb dedicated internet transit connection. This dedicated connection allows for re-sale and isn’t oversold to many different customers. The backhaul costs can vary a great deal based on the remoteness of your location and other political factors, but we will work with a cost of $1500/Gb for our analysis here.

With just 100 users, the business can expect a gross revenue of $4600, and a net of of $3100, an average of $31/user. In many ISP businesses this initial build out to 100 subscribers can take about a year, however, with our Altheahoods program, you can easily front load the user acquisition into the planning stages, and build out with 100 subscribers initially.

For this build out, both the gateway node and the subscribers will need equipment. The average cost for the CPE (customer premises equipment) is between $150-$200, and this includes a router, the antenna, and cable at the subscriber's home or business. Organizers can choose to use a model where subscribers buy their own equipment, organizers pay for the equipment of the subscribers outright, or organizers pay for the equipment up front but increase the monthly subscriber fee for a short time to cover. For this example, we’ll look at the most costly use case for the business - organizer paying for all equipment up front.

Below is a bill of materials for a 100 person network with one gateway and three relays. With the total cost of the build out, including all gateway and relay nodes and all subscriber CPEs, being $19,881 and a net of $3100 a month, ROI can be expected in just under 7 months.

Of course, we are overlooking labor for installers and other costs, like insurance. These costs can be managed by increasing the length of time to ROI, charging an installation fee, or the organizers can install themselves. Once the initial build out has been completed, typical growth for wireless ISPs can be 15-30 subscribers per month, adding additional revenue. The 1Gb backhaul gateway could support between 200-400 subscribers, depending on expected throughput, until it either needs to be upgraded or more gateways built. Althea supports a sustainable implementation and scalable business model, with a short term return on investment.

Cooperative or NonProfit

The cooperative or non profit model is very similar to the business case, with slight differences to how costs can be shared. In the rural Oregon Clatskanie network, the organizers formed a member owned cooperative, where subscribers are members and the network is governed by an elected board. The cooperative both organizes the network and operates the main gateway.

The cost for the CPE equipment, gateway, and relays is the same as the above business case. Backhaul costs $1400/month for 500Mbps, which can support about 100 subscribers with speeds of around 50Mbps.

The Clatskanie build for it’s thirty-five users was primarily subsidized by Althea as testing grounds for our new software. This funding could be similarly achieved through a broadband grant or crowdfunded. A shared approach might look like relays that are self funded and the gateway and CPE’s equipment being a shared cost. At thirty-five users and a total cost of $7106, that would be $203 per person and easily within the scope of a crowd funded project for an under served area or one with little choice for internet access.

At 100 users, the network will be grossing approximately $4600/month and a net of $3200 which will be able to support a part-time manager/installer and the other expenses of the network. Growth can be easily scaled as more bandwidth and subscribers are added. This creates an inclusive organization able to sustainably support broadband expansion in a variety of environments, including low density or underserved low income areas.

It's June 2019 and today I'm going to install a new subscriber of our cooperative ISP in rural Oregon. This family had previously only access to 1Mbps wireless or an expensive 4Mbps DSL connection. This is not a new or isolated incident. I receive calls every day from people in my community that are desperate for broadband so they can access education, health care or work online. So, when the FCC posted the results of the Broadband Deployment Report, which was titled “Digital Divide Narrowing Substantially” and portrayed a rosy picture of internet access, we had good reason to be suspect.

The FCC and big telecom have strong incentives to show America that their policies are working, and that competition and municipal solutions aren't needed to improve access. The FCC collects the data for the Broadband Deployment Report from numbers that internet providers self report. There are strong incentives for providers to be overly optimistic about the access they provide. If an ISP in the area doesn't claim to provide access to all the nearby census blocks at broadband speeds, other carriers can receive grants to build and encroach on their “territory”. The FCC leadership also has a strong incentive to show that their vastly unpopular policies work - that allowing ISPs free reign to block and throttle data somehow translates to providing access to rural and under served areas.

A regulatory body that hears mostly from lobby and special interest groups incentivized to overstate access could benefit from hearing from the members of under served communities. I would like to extend an invitation to FCC chairman, Ajit Pai, to visit Clatskanie Oregon and hear from farmers, business people, and young students who are directly impacted by lack of broadband access. These conversation can curate a deeper understanding of the needs of rural communities and impact policy in a more meaningful way than self-reported carrier data. We look forward to the opportunity to share our first hand knowledge and engage in dialogue that can lead to real solutions to bridging the digital divide.

Building your own blockchain also gives you a lot of flexibility. If Bitcoin is a pocket calculator with a few functions, and Ethereum is a graphing calculator with the ability to upload some limited scripts, then Cosmos is Ruby on Rails. You start from a full-featured base application built from a robust set of libraries, and customize anything and everything about it.

The base application that you start from is called gaia. This is the same software that the Cosmos Hub chain runs. This blog post gives instructions for starting a chain with 2 validators, although it would be easy to adapt these instructions to a larger group of validators.

Install

First install everything. These steps are the same if you are initializing a new Cosmos chain, or joining someone else's. They work on new DigitalOcean Ubuntu droplets, but will probably work on many other Linux systems, although you may have to install other dependencies such as Git.

Download and install Golang (from https://golang.org/doc/install)

wget https://dl.google.com/go/go1.12.5.linux-amd64.tar.gz
tar -C /usr/local -xzf go1.12.5.linux-amd64.tar.gz
export PATH=$PATH:/usr/local/go/bin

Set up some required Golang environment variables (from https://github.com/cosmos/gaia/blob/master/docs/installation.md)

mkdir -p $HOME/go/bin
echo "export GOPATH=$HOME/go" >> ~/.bash_profile
echo "export PATH=\$PATH:\$GOPATH/bin" >> ~/.bash_profile
source ~/.bash_profile

Get some build tools used to build the binaries

apt install make
apt install gcc

Download gaia, the Cosmos hub codebase

mkdir -p $GOPATH/src/github.com/cosmos
cd $GOPATH/src/github.com/cosmos
git clone https://github.com/cosmos/gaia

Build the binaries

cd gaia && make install

Check that it was installed correctly

gaiad version --long
gaiacli version --long

gaiacli for instance should output something similar to:

cosmos-sdk: 0.33.0
git commit: 7b4104aced52aa5b59a96c28b5ebeea7877fc4f0
build tags: netgo ledger
go version go1.12 linux/amd64

Initializing a new Cosmos chain

Now it's time to set up the starting conditions for your network using a file called genesis.json. This is very important on a proof of stake chain, since everything else that will happen during the life of the chain will originate from the authority and value granted here. For example, on the Cosmos Hub, the genesis.json file contained the amounts of Atoms purchased by crowdsale participants.

The process of starting a Cosmos chain has multiple steps, but each step is important. Participants in a blockchain system trust the process instead of each other, and each step of this process lets them verify that everything is kosher.

1. Someone creates a genesis.json file with initial token allocations and sends it to all the starting validators for inspection.
2. If the validators feel that the genesis.json file is valid, they sign gentxs to get added as validators on the chain.
3. Someone compiles all the gentxs and adds them to the genesis.json, again sending it out to all the validators for inspection.
4. If the validators feel that the genesis.json is still valid, they start their nodes and start producing blocks.

We'll be using the instructions here for a "Single-node, Local, Manual Testnet", but we will modify them to create a multiple-node, remote, manual network. This assumes that you've got two servers set up according to the above instructions. If you only set up one, go ahead and set up another one now.

Step 1.

Follow these instructions for validator 1:

# These commands are run on validator 1

# You can run all of these commands from your home directory
cd $HOME

# Initialize the genesis.json file that will help you to bootstrap the network
gaiad init --chain-id=mychain validator1

# Create a key to hold your validator account
gaiacli keys add validator1

# Add that key into the genesis.app_state.accounts array in the genesis file
# NOTE: this command lets you set the number of coins. Make sure this account has some coins
# with the genesis.app_state.staking.params.bond_denom denom, the default is staking
gaiad add-genesis-account $(gaiacli keys show validator -a) 1000000000stake,1000000000footoken

Now, switch to validator 2

# These commands are run on validator 2

# Create a key to hold your validator account
gaiacli keys add validator2

# Show the address
gaiacli keys show validator2 -a

Copy the key output by the second command and switch back to validator 1:

# These commands are run on validator 1

# Add validator2's address and allocation to genesis.json
gaiad add-genesis-account <address copied from validator2> 1000000000stake,1000000000footoken

# Export genesis.json
gaiad export

Copy the genesis.json output by the second command and switch back to validator 2:

# These commands are run on validator 2

# Save the genesis.json from the last step in:
$HOME/.gaiad/config/genesis.json

Step 2.

Now it is time to make the gentxs

# These commands are run on validator 2

gaiad gentx --name validator2

# This should output something like:
# Genesis transaction written to "/root/.gaiad/config/gentx/gentx-33e8c0fbef25d8d455aefa1fd33fb6982f5484ec.json"

# Copy the gentx from the file where it was saved

Step 3.

Switch back to validator 1

# These commands are run on validator 1

# Save the gentx from validator2 to:
# /root/.gaiad/config/gentx/<name of gentx file from validator2>.json

# Now make a gentx file for validator1 as well
gaiad gentx --name jehan

# Now generate the final genesis.json
gaiad collect-gentxs

Switch back to validator 2 and again save the genesis.json file produced by validator 1

# These commands are run on validator 2

# Save the genesis.json from the last step in:
$HOME/.gaiad/config/genesis.json

Step 4.

Now, running gaiad start on both nodes should start your blockchain. You may have to start one or the other one first. You'll probably want to use something like systemd to keep the validators running when the system restarts, but that's beyond the scope of this blog post.

In the next post, we'll talk about how to send coins to another account, and how they can get added to the validator set.

How a poorly coded smart contract disrupted an ISP in rural Oregon.

Around 3pm this Sunday we started experiencing degraded service for the entire Clatskine test network. An automated alert notified the on call network operator and the investigate commenced.

A few minutes of fervent log searching later a pattern emerged. transactions weere being published to the Ethereum full nodes successfully but not a single Althea transaction had made it into the blockchain for the last 5 hours.

While the bandwidth billing system is very resilient to temporary failures in payment it does have to cut off users who don't pay eventually. In our case the gateway determined it was being underpaid by the exit and started limiting the connection speed to the exit as an underpayment warning.

The bigger question of course was, why were the exits payments not getting through?

The answer to that comes in two parts. A poorly coded airdrop script and someone who was exploiting it.

You see this airdrop script suffered from a vulnerability we call a 'sybil attack' it allowed anyone with a small amount of Ethereum in their account to mint some small amount of this new token for free.

An enterprising hacker simply wrote a script that would take a large amount of Ethereum, split it into many new addresses with smaller balances and then call the contract. Allowing them to collect many hundreds or thousands of times the amount of minted token they were supposed to be limited to.

This script interfered with Althea specifically because both Althea and the hacker selected the same method for determining the gas price on our transactions.

The script was producing thousands of transactions with a 1Gwei gas fee, taken from the gas_price() endpoint of an Ethereum full node. That value is computed as the median of fees paid over the last several blocks.

Althea uses the same endpoint and there is flaw we were looking for. Since we chose the same gas fee as the contract attacker, our transactions had the same priority while being hundreds of times less prevalent. Resulting in almost none of them getting through.

If we had increased our tx fee value by even 10% Althea transactions would make it into the blockchain in less than 3 minutes. But because we trusted the median gas price endpoint our transactions were directly competing with the attacker and would take over two hours to get in a block, if they got in at all.

We patched our nodes to pay 50% more for gas and had the network back up and fully operational within 15 minutes of the first service degradation.

The Althea routers advanced traffic shaping kept the network slow but usable for the duration. We actually ended up with many more complaints about this weekends Google Cloud outage.

But the problem here is not over. The fix required human intervention and the level of 'meta' thinking to determine what the solution would be.

Althea is unusual in that autonomous micropayments for bandwidth have higher expectations on the level of automation. People shove networking gear into a cubbord and leave it there for years unattended. No one is going to come along and refresh the page or reboot the device.

In order to achieve that reliability we've slowly been building more and more awareness of the all the failure modes of blockchain payments into our own software. Modern blockchains are simply not reliable enough to treat as an abstracted payment system where a transaction is valid, submitted, and done.

Hopefully proof of stake based systems can reduce the variability of the transaction process as well as help reducing scaling issues. In the meantime we'll need to update Althea to inspect transactions in the mempool and automatically make the gas price adjustment required to win the next duel.

Beta 5 Released!

The Beta 5 release is out with some new features, the real headliner is that routers now estimate how long your existing balance will last based on previous usage.

• Client devices no longer come with WAN ports by default
• even more fixes to payment stability
• bandwidth history and payment history now stored and displayed on the dashboard
• changes to client billing that make it resilient to packet loss
• fix issue with binding to ports not on built in switches
• more fixes to port selection for peer discovery
• protection against payment replay attacks
• fix to babel parsing issue that could cause strange billing behaviour

As you can see I need to top up my personal router

On a sunny day in the city of Los Mochis, Mexico, Althea organizer Raul, runs a speed test on an expensive 20Mbps connection. 20Mbps, which is not even considered broadband in the US, is about the fastest connection available in the city. Today, like most days here, the speedtest shows an underwhelming 1Mbps of download speed. Raul is here to change this - to bring real improvement to both the cost and speed of internet, starting in Los Mochis and growing to all of Mexico.

By selecting a quality tier two connection and using Althea software, Raul is planning on offering a true 25Mbps connection at a target cost of $20 US dollars per user. This will give people access to video streaming, online commerce and education that most can afford.  In addition, Althea's incentivized system will allow Raul to work with local partners to become relays, so the money generated from the network will stay local. These relays will help spread the word, and earn money from the bandwidth of those connected to them.

In May, Raul set up a gateway and small test network, sharing the Althea system with local people and working with local installers. A gateway is a connection to the internet that is then broadcasted with outdoor antennas to the community. People here are excited about Althea and are ready for better internet access at more affordable prices. In July, Raul will travel back to Los Mochis, this time ready to build the network. Faster, cheaper, locally run internet is coming to Los Mochis and Mexico!

In the end of May, another Althea organizer will be deploying a pilot program in the city of Los Mochis, Mexico. Raul hopes to start with connecting around 25 people and then quickly grow and expand into other Mexican cities. He is also planning to connect some of the nearby schools, who lack consistent internet access. Because Althea can work with a variety of hardware, he plans to use low cost routers and antennas to keep installations cheap and accessible for most people.

We also recently checked in on Tamario and the other participants in the Medellin Althea deployment. While there have been some setbacks, the Red Medellin team is working through challenges and pushing forward with their plans to make exchanging pesos for xdai easier. Althea infrastructure provides much faster home internet access, as Tamario demonstrates in this  fun video.

We continue to grow into a reliable network in Clatskanie, adding a few new members weekly. (read about some of the people involved in Oregon Business) Recently we have demonstrated that even with two layers of encryption and all of our software, participants can expect the same, or even better, performance than they would get with a legacy ISP. Justin Kilpatrick takes us through all of the tech achievements in the latest blog post. There is a lot to celebrate, and, as he puts it, “we nailed both performance and price”.  With those milestones reached, we now focus on adding new users, tweaking the economics, and flawless reliability.

On Wednesday May 22nd, we shared these milestones in more depth and answered your questions in our second live community call. Check it out here. In June, Jehan Tremback will be visiting Berlin for the Cosmos conference, Interchain Conversations and Althea CTO, Justin Kilpatrick, will be speaking at the NANOG conference in Washington DC.

If you are interested in becoming an organizer, starting an Altheahood in your area, or have questions, send us an email or check out althea.org.

One young person in the Comuna, Tamario, hopes to make a living off of the internet via his Youtube channel filled with quirky videos of daily life. Like his distinctive bi-color hair style, Tamario's videos are fun, and creative - a combination of humor, music and dancing. Before Althea, he had no home access to internet, and would have to rely on a friend's slow DSL connections to upload his videos.

When Tamario was introduced to Althea's technology by members of the Red Medellin team, he was eager to help. Tamario became a key community member who not only helped build the network, but also became a relay, providing nearby homes with internet. Tamario will make a few cents a GB from each neighbor connected to him, which will pay for the electricity required by the antenna and some extra.

This project is still in it's early stages and experiences some problems that were new to us at Althea. We didn't realize how expensive power consumption can be for the people who live there. The crypto ATM downtown is also far away and difficult to use. We have been working to make the process easier, with our simple wallet, www.efectivo.cc, and an ethereum to xdai bridge in the routers. The Red Medellin team is working with local grocery stores, the tiendas, to provide liquidity and access to xdai within the Comuna as well.

Internet access that's too expensive for the community or charges more for some sites than others, is not really access at all. Althea's agile, decentralized network is helping people in the Comuna experience true access to the internet, which means people there can now use the internet for education, staying in touch with loved ones, and Tamario can share his videos with the world.

This is a node in our test network, these packets are being routed through a point to point antenna over a distance of 3 miles before reaching the gateway and peering to the internet through the exit.

root@OpenWrt:~# ping 1.1.1.1
PING 1.1.1.1 (1.1.1.1): 56 data bytes
64 bytes from 1.1.1.1: seq=0 ttl=61 time=5.529 ms
64 bytes from 1.1.1.1: seq=1 ttl=61 time=7.793 ms
64 bytes from 1.1.1.1: seq=2 ttl=61 time=7.339 ms
64 bytes from 1.1.1.1: seq=3 ttl=61 time=5.999 ms
64 bytes from 1.1.1.1: seq=4 ttl=61 time=7.361 ms
64 bytes from 1.1.1.1: seq=5 ttl=61 time=7.839 ms
--- 1.1.1.1 ping statistics ---
6 packets transmitted, 6 packets received, 0% packet loss
round-trip min/avg/max = 5.529/6.976/7.839 ms

Lets beak this down. Cloudflare has done it's best to put an endpoint in every internet exchange in the US. This makes 1.1.1.1 the best way to compare last mile latency since the ping packet will only traverse the distance to the datacenter and back.

Here's a sample from a traditional cable ISP.

[justin@localhost ~]$ ping 1.1.1.1
PING 1.1.1.1 (1.1.1.1) 56(84) bytes of data.
64 bytes from 1.1.1.1: icmp_seq=1 ttl=56 time=4.58 ms
64 bytes from 1.1.1.1: icmp_seq=2 ttl=56 time=8.43 ms
64 bytes from 1.1.1.1: icmp_seq=3 ttl=56 time=7.79 ms
64 bytes from 1.1.1.1: icmp_seq=4 ttl=56 time=9.90 ms
64 bytes from 1.1.1.1: icmp_seq=5 ttl=56 time=6.88 ms
64 bytes from 1.1.1.1: icmp_seq=6 ttl=56 time=4.59 ms
--- 1.1.1.1 ping statistics ---
6 packets transmitted, 6 received, 0% packet loss, time 5005ms
rtt min/avg/max/mdev = 4.587/7.033/9.909/1.950 ms

And a Gigabit fiber connection.

[justin@DESKTOP-UALBV95 ~]$ ping 1.1.1.1
PING 1.1.1.1 (1.1.1.1) 56(84) bytes of data.
64 bytes from 1.1.1.1: icmp_seq=1 ttl=55 time=12.1 ms
64 bytes from 1.1.1.1: icmp_seq=2 ttl=55 time=11.7 ms
64 bytes from 1.1.1.1: icmp_seq=3 ttl=55 time=12.1 ms
64 bytes from 1.1.1.1: icmp_seq=4 ttl=55 time=11.9 ms
64 bytes from 1.1.1.1: icmp_seq=5 ttl=55 time=11.9 ms
64 bytes from 1.1.1.1: icmp_seq=6 ttl=55 time=13.2 ms
--- 1.1.1.1 ping statistics ---
6 packets transmitted, 6 received, 0% packet loss, time 12ms
rtt min/avg/max/mdev = 11.677/12.145/13.231/0.520 ms

This sort of performance strains the bounds of credulity. It's the sort of thing that makes you double check you're on the right connection.

We've got two layers of encryption across three nodes, for a toal of six encryption operations. Two NAT operations, three routing operations, and three traffic flow control operations.

And the wireless point to point hop between this node and the next.

How is this even possible? The answer is a combination of two factors.

Traditional ISPs don't really apply new technology very much. They update their modems for new wires but are very conservative with the networking software.

Althea has taken the same base software ISPs use and applied some modern advancements.

• All of our packet forwarding is in kernel, this lets us use the same highly optimized code relied upon by the rest of the industry. But we're using a version a couple of years ahead of what's in the average ISP modem.
• The Cake flow control algorithm dynamically prioritizes low latency traffic, preventing other Althea users from slowing down packets while at the same time ensuring efficient sharing of connections. This is probably where our latency advantage comes from.
• WireGuard has hand tuned vectorized implementations of it's cryptography, encrypting or decrypting a packet takes only a few clock cycles. This minimizes the impact of the encryption between nodes.
• We racked our own exit server at the PDX data center. Mere feet from Cloudflare's 1.1.1.1 server. This means that the 'exit sever' part of Althea's archietecure does minimal damage to latency and throughput.

In short we've stacked our advantages and minimized our overhead to such a degree that we come out even with centralized ISP performance.

These cheaper ($65) antennas typically go up to about 100mbps.

The entire set of hardware for this user cost $161, $90 for the router, $65 for the antenna, and $6 for the mount.

I'm very happy to have reached this design milestone. We knew that to be viable as a replacement for centralized ISPs we would need to nail both performance and price.

But the journey is far from over.

What we need to improve from here is reliability, we currently have 99.7% uptime, that's about 30 minutes of unexpected downtime a week.

Obviously this is a user experience issue, even if we can provide better service than an incumbent ISP we need to also be more reliable.

Payments have reached a high level of reliability

Since the last update my primary focus has been getting payment convergence to hold up in turbulent production conditions.

I've been hotpatching the production exit and gateway every day for the past two weeks. There are obviously risks to doing this, but I've managed to get away without any extra downtime beyond what it takes to actually load the patches.

This is the result of Saturday's hotpatch, 0.3% bandwidth accounting divergence and 0.6% billing divergence over a period of 48 hours. Including some of the most active traffic periods of the week.

Before I started this billing would drift by a couple of percent a day and on the larger nodes errors would eventually build up enough to stop normal operation.

The core set of problems that ended up requiring changes are as follows

• Receiving nodes checked for 1 block confirmation so uncles could cause issues. Increased to 4 blocks
• Sending nodes assumed that all transactions that entered the mempool would eventually enter the blockchain, this is not the case. Sending nodes now also wait for 4 confirmations.
• The state of 'payment in flight' was implicit. So sometimes a node would make many transactions during the same block period. This is now prevented.
• The timing issue I resolved in my last blog post, no more time based discounts.
• The accounting logic for payments didn't deal well with high rates of failed payment attempts. This logic has been simplified and made aware of in flight payments.

I'm hoping to roll these changes to everyone in Beta 5 in just a week or so.

The billing disagreement mystery

For the past several weeks I've been working on figuring out why billing amounts on production routers would always diverge eventually.

All the routers would agree on who should pay who what for a few days, maybe a week. Then things would start to drift.

Routers on each side of a connection would agree about how much had been paid, but disagree about what was the correct amount.

It obviously couldn't be a simple logic error. Controlled testing stubbornly produced perfect results, even if run for days on end.

I considered and investigated the possibility that packet loss was producing the billing drift in production. But that was never borne out by the data.

After a few weeks of looking for stupid mistakes I had to accept that the problem I was seeing might be more fundamental. In the end it was a different bug that put me on the right track.

On Thursday the 18th at around 6pm everyone in our test network was just getting home from work and internet usage was at its non-weekend peak.

Things seemed to be ticking along smoothly, when system lag on the gateway suddenly began spiking. Seen here in the monitoring system we use for our infrastructure.

At first only a little and then 30 minutes later it began rising without control.

Google's dns server 8.8.8.8 was behaving strangely in the Portland area. The server would accept connections but once open they would hang.

Being a multihomed major service 8.8.8.8 is not provided by a single server but a network of different machines around the country. 8.8.4.4, the fallback server, seems to have been just fine during this period.

Once the incident was over the system self recovered within 20 minutes or so. The post mortem was completed with a single line fix.

This issue got me thinking about the relativity of time on the routers. Our billing algorithm made the assumption that everyone did billing computations every 5 seconds, what would happen when that didn't hold true?

Could small disruptions in the sampling frequency be causing our divergence?

Some quick data gathering showed that a 1 second (20%) variance in average sampling frequency was not only happening but nearly universal.

Breaking the rules

The core of the problem was that Althea's billing was comparing two curves.

One of these was a flat line, the discount for 'free tier bandwidth' the other was a sampled curve, like the graph below each router would see a different curve depending on when and how often it sampled.

Because the free tier discount worked a 'discount per second' on bytes used it links the amount paid to the accuracy of the sampling mechanism.

Over time these differences would add up as the various routers in the field where exposed to different input.

This sort of problem is fundamental sampling theory, we need to sample at at least the Nyquist rate in order to produce an accurate curve.

There's no way we can sample anywhere near that fast, nor can we guarantee a agreed-on slower schedule will be followed by all routers.

If the user decides they need to start streaming 4k Netflix the operating system will put billing on hold to deal with the traffic spike. There's nothing our billing process can (or should) do to stop this.

Thankfully there's a pretty simple solution, in order to prevent the sampling frequency from causing problems we need to avoid involving any time-denominated metrics in billing.

Bytes used is simply a scalar metric, one that will count up to be the same if sampled once a day or once a microsecond.

Sadly this means the free tier billing discount had to go.

What does this mean for users?

In terms of networking the free tier is staying around. Relays provide 1mbps without any charge or conditions so that users will always be able to acquire and deposit more money to pay for bandwidth. It makes sense for the network and it makes sense for the relays.

We've adjusted the prices in our test networks to ensure that this change has a very minimal impact on the users bills during normal operation.

A user determined not to pay for their internet could in theory use a couple of dollars a month of bandwidth from their upstream. Previously this would have been zeroed out by the free tier discount, but now it must come out of the earnings of the upstream relay.

This isn't really a concern in our current networks as no one runs on empty for very long at all.

But over the course of several months or even a year the total amount of 'stolen' bandwidth is significant enough that we'll have to figure out how time box use of the 'free tier' bandwidth.

What's new in Beta 4?

• Changes to billing rules to prevent divergence over long periods
• Replaced individual wifi settings endpoints with a single one, to make setting up wifi while on wifi smoother
• Added simplified dao payments (just pays into a multisig)
• Fixed a bug where port allocation would recurse forever in some cases
• Fixed a bug where price display would overflow in Babel
• Fixed a bug where gateways would hang if the dns server misbehaved
• Fixed a bug where 15 minutes after port toggling all other mesh ports would cease to work
• Babel now takes into account the RTT of various paths

Taking a note from Google's rollout of their fiber networks, we are introducing the new Altheahoods program! This program supplies organizers with the marketing tools and internet presence to get end users signed up and on board prior to building out the network. This is a good tool to evaluate if an Althea network will be viable in your neighborhood, and requires very little investment to get started. Shoot us an email at hello@altheamesh.com if you are interested in bringing a faster, cheaper, and decentralized Althea network to your neighborhood.

We recently rolled out a major dashboard update and text notifications for a low balance state in the routers. Justin Kilpatrick, CTO, weighs in on this and other development updates.

“The new dashboard and text notifications are the latest user facing features, with a focus on helping users have a smooth experience as we move into requiring and enforcing payments in our test networks. Behind the scenes we have continued to work on stability and performance improvements. Our new exit server architecture is significantly more resilient and client router failures are down from previous months. We are also testing an Android app to allow organizers to easily rescue broken routers in the field without a bulky laptop."

As many of you know, Cosmos has now launched and we are moving forward with development of our own cosmos side chain that will allow users to more easily use stablecoins as the base currency. Payments on this blockchain are done using the stablecoin dai, which maintains its price relative to national currencies so the user's deposit to their router remains stable. This blockchain will be maintained by validators running the Althea blockchain software. These validators are given power to validate the Althea blockchain by holders of the Althea governance tokens, who delegate tokens to the validator(s) of their choice. Validators and those delegating Althea governance tokens to them earn transaction fees from each payment. You can learn more about ALGT in our whitepaper.

Our first live community call is coming up! It's the perfect opportunity to get the latest Althea updates and have all your questions answered. We will be live on YouTube April 24th, 1pm PST. Sign up here and let us know what you want us to talk about and your questions. Looking forward to seeing many of you there!

Beta 3 is finally rolling out, after several weeks of furious QA. Here are some example bugs.

1) Fixed a race condition where the Linux kernel won't assign fe80 local ips to interfaces despite completing the 'network' boot step. This occurs for no more than one in several dozen reboot cycles.

2) Fixed a bug where after about two years power on time (across the fleet of devices) the local server will respond improperly to requests. The fix for this is actually the same as the fix to the lack of recovery bug we mentioned in the last update.

There's a good reason why the basics of home networks haven't changed in decades. Debugging a program running in an uncontrolled environment out in the field requires a huge amount of intuition and often more than a little luck.

I only figured out what was going on with (1) because I stumbled on a machine that did it every single time and (2) is a bug we've been seeing in one form or another for months, we just finally caught an instance that gave us just the right data to reach conclusions.

It's great that we're fixing these sorts of long tail issues, it means we're getting close to the point where we can take Althea out of beta.

Entonces(enˈtonses/, [ẽn̪ˈt̪õnses]) — One of my favorite Spanish words. It means then or so. I like the cadence, how it fills the awkward space in a sentence and how it conveys a sense of time and place. My trip in Colombia was a journey through time and place, the logistics of project management — The first this, then that and the witnessing of a unique place at a unique time in history. How Medellin, a once violent place, has transformed itself and revitalized its economy into a Mecca of progressive infrastructure and tourism.

Day 1: Where is the Box?

Medellin sits in a valley at the elevation of 4,905 ft, surrounded by dark green mountains. Housing and communities are haphazardly built up from the urban center along the sides of the mountain, reminiscent of barnacles clinging to the rocks on a seashore.

I arrived after an all night flight and was greeted by our gracious hosts, Forrest, Andrew, and Kelsey, the red_medellin team. red_medellin is a group of American ex-pats that have coordinated with the accelerators, non-profit organizations and the local community here using Althea to bring internet to the much needed outlying poor communities. We took a winding taxi cab ride down the mountain into Medellin. Driving here has a video game quality — everyone is seemingly unaware of traffic laws, acceptable speed, and dividing lines. That’s when I saw it — an elusive horizontally aligned sector antenna! In most places, sector antennas are aligned vertically so that the beam covers the largest area possible horizontally. A horizontally aligned sector would provide coverage vertically, so it would be useful only on the sides of a steep hill or coverage to a skyscraper. I suspect there are very few places in the world where this makes sense.

I brought with me a duffel bag full to the brim of routers, cables, and an old Thinkpad for programming radios. It’s fortunate that I did so, because the box that was shipped from the United States had not yet arrived. Neither had the boxes with Ubiquiti radios, ethernet cables, and most everything else we needed to build the network. There were other logistical issues that had manifested as well. The location for the sector antennas, a school perched on the very top of the community, has not been confirmed and Claro, the ISP providing our backhaul connection, has not yet told us when they will be installing. All was not lost, however, there were still routers to be flashed! So, we began to flash routers with the Althea firmware while Andrew and Kelsey made phone calls. The phone calls were successful and the radio equipment arrived without incident and we made a tentative appointment with Claro. As I walked back that night, I heard the band playing — Bon Jovi’s “It’s My Life”, sung in heavily accented English, from a Hard Rock Café.

Day 2: Shopping

The red_medellin team has chosen Casa Morada, a location nearby the Comuna, to beam the wireless backhaul to a school at the top where the sector antennas will broadcast internet throughout the community. We will need a roof mount for the RocketDish antennas chosen for Casa Morada. Jehan, who’s also working on Althea, has arrived this morning and shown some skepticism that the rocket’s antenna, a 31dbi dish which looks quite large in its cardboard packaging, is perhaps overkill for this less that 5km shot to the school. He communicates this observation through subtle chuckles and ponders whether the smaller NanoBeam antenna may have been adequate.

Andrew, Kelsey and I go to take a look at the roof at Casa Morada, and it’s determined that a non-penetrating roof mount, will do the trick. Where do you buy a non-penetrating roof mount in Medellin, though? The typical way of buying things online doesn’t work here, and even in the United States it is a specialty item not available at hardware stores.

We decide to head to the market areas of town, a place with streets of many small vendors. The vendors here typically have a small area to display their merchandise and hand you the items from behind counters. And the carts! Many sturdy, well-made carts are loaded down with fruit, vegetables or other wares and parked or rolled up and down the street. Some vendors, weary of calling out the name of their goods, have pre-recorded advertisements played on antiquated loud speakers. I’m reminded of the cart in a popular Colombian music video, one laden down with a 70s era speaker system and rolled down the hill behind dancing people.

We stop at the hardware vendor, who points us three blocks down to a sort of industrial section of town. We pass restaurant equipment, motorcycle repair shops, and finally see the welding sparks we were looking for. We show the welder a picture and Andrew, who speaks fluent Spanish, helps negotiate price and delivery. He can manufacture and deliver the stand tomorrow, so our work here is done. As we walk to a cleaner place to eat, a hat vendor keeps pace with us, in case we may change our mind about a hat purchase today, and there in a courtyard, I see a man in full armor, waiting, I surmise, for someone to take a picture and give him a tip. But, why, I wonder? Does full medieval armor have any cultural significance? He seems very out of place as he stands alone in the heat. What led him down this occupational path? I wish I spoke Spanish — I would have asked.

In the afternoon, we make our first trip to Comuna 13. Comuna 13 used to be a very violent place; a place controlled by Parmilitares, FARC, and ELN because of its strategic location suited to the transport of cocaine and weaponry. Recently, the government has made many strides to improve infrastructure and transform Medellin into a what is now a popular tourist destination. The transformation of Comuna 13 is something Medellin is proud of, and there are even tourist “graffiti tours” in another part of the neighborhood, where the government has installed escalators. Something about a bunch of rich tourists on some glitzy tour of how poor people live doesn’t quite sit right with me though.

The taxi cabs stops at the bottom of the hill at an intersection and a place with a few small grocery shops and restaurants. Jehan and Forrest are there with most of the radio equipment and cabling that will be brought up to the Son Bata recording studio, which we are using as a base camp and secure place to store the equipment. And then we hike, up the steep incline of the main street and up multiple staircases and inclines. Most of the people here are under the age of sixty, and we see signs of daily life as we walk — a hair dresser is braiding hair, children are playing, and music comes from one of the many houses piled on top of each other.

We hike to the top of the Comuna where the small school is located — escuelita, I hear one of the children call it. The view from here is breathtaking. The dense housing, strong community and the incredible line of sight make Comuna 13 an ideal place for an Althea wireless network.

Day 3: Sick

I am sick to my stomach all night and this day is wasted time for me.

The rest of the team, together with a local handy man, installs the roof mount and the RocketDish at Casa Morada. They head back to the Comuna and hold a community meeting. There are about fifteen attendees at the meeting. Many here are people wishing to be an intermediary node, which is a house that connects to other houses nearby and earns some money from forwarding bandwidth along. Here in the dense housing environment, these homes will be connected with an outdoor rated ethernet cable. The team also installs the first client node at Tomario’s house. Tomario quickly takes to learning how to crimp networking cables and installing the LiteBeam antennas. It’s interesting to envision how this group of local people building and supporting the network, a subnet DAO, might coalesce here.

Day 4: A late start

Forrest and I got a late start but did manage to do some shopping for supplies, flash the EdgeRouter x for use at the school, and set up the the PrismStations (the sector antennas).

Day 5: The school

We went back to Comuna 13 again. We were hoping this time to get a definitive answer from school staff that we could locate the receiving end of the wireless backhaul and the sector antennas here. Up to this point, many phone calls and conversations have gone back and forth, attempting to determine just who had the final say in the matter. Many of the critical elements of this project come down to getting the “yes” from the appropriate person. The “entonces” moment — the “yes” from the right person, and then we can move forward.

When we get to the school, we are granted an audience with the school administrator and Andrew patiently explains the goal of our project and the mechanics of how we would like to mount the antenna. Many small school children flock around us, asking questions and practicing their best “hellos” in English. While the school administrator likes the project, she needs to see more diagrams and a written description of the mounting plans. There is still time in the day to do more of the LiteBeam installs. The LiteBeam is a smaller CPE radio, or customer premises equipment, for use in this case at intermediary nodes to connect them with the main sector antennas at the school. We first hook up the sector antennas temporarily at Son Bata so we can bridge them and show the home owners how they can aim the LiteBeams. Most of the people living in Comuna 13 own their homes, and I’m told a home here sells for around $5000.

We do a quick setup at one of the more colorful homes, set up high on the hillside where we find the antenna almost impossible to not aim correctly with such perfect line of sight. Jehan and I have to leave earlier than the rest of the group for an interview at Espacio, a local media incubator, which means we have to attempt to catch a taxi. This proves to be a rather difficult and we walk for some distance before finding success. This gives me the opportunity to spot some horses tied up at a nearby school and a poster which lists the prices for internet. red_medellin plans to provide Althea internet to the Comuna for 600 COP a GB, far lower than the current ISP’s 27,000 COP for 1.3GB. The day ends with a trip to the Blockchain Center and Andrew gives a presentation in Spanish about Althea to local Ethereum enthusiasts. There is free pizza offered here, just like Ethereum meetups in the United States.

Day 6: The Last Day

It’s my last full day in Medellin and we have heard that Claro might be installing the backhaul internet at Casa Morada today. We head there, hoping that is the case and that we can finish installing the Althea router. The router chosen as the Althea gateway is, in fact, just an older Dell Optiplex that has been flashed with the Althea firmware. It’s perfect for this location because we have a bit more room and we can use the extra performance a desktop can provide. Unfortunately, Claro did not keep their appointment time, or call us to let us know they would not be coming. We did, however, have a great conversation with some local young people, one of whom was holding an injured bird in his hand. Andrew and Kelsey also took the opportunity to share with me the amazing public transit system and we took a tram ride over another part of Comuna 13. Medellin’s public transport system is a source of pride, as indeed it should be — it’s kept clean and orderly and dozens of people direct traffic and keep things moving forward.

I am excited to see the progress as red_medellin continues to move things forward with their Althea deployment. The yield of their efforts will be faster, less expensive internet for the community that will provide education and economic opportunity. The financial benefit of this system will also be kept in the community instead of benefiting large, monopolistic ISPs. “Entonces una mejor internet para todos.” Then, a better internet for all.

Thanks to Jehan (hide) and Justin Kilpatrick (hide).

People in Clatskanie are now paying for their internet by loading Ethereum on to their routers using the Coinbase mobile app with a debit card and scanning the QR code on the router dashboard. They are happy to leave behind their legacy providers and make the switch to faster, cheaper Althea internet.

Linda, whose farm is a relay node and also uses Althea at her business in town remarked, “The internet we get through Althea is so fast - lightning fast,” she said. One night my husband asked everyone in the family to stream video at once to see if it would slow down and we were surprised to see that it was still going strong. “I’m just in awe of the speed!”

Althea team member, Sebastian Bolanos, recently traveled to Colombia to meet with Althea users. The wireless link between the fiber connection towards the center of town and the school on the high hill in the Comuna was established. Now the members of the community are enjoying fast Althea internet. Sebastian described how people were eager to try out YouTube with their new Althea connections, and were excited when they could play their videos without buffering. Colombians are paying for their internet with the xdai stable currency and our custom made wallet, efectivo (https://efectivo.cc/).

We are currently putting the finishing touches on a big re-design of the router dashboard. In the coming software release, we also added failover for exits and a “low balance” text notification feature.

You can keep up on all the technical details of our releases here on the Althea blog.

A big component of the Althea system is supporting our local organizers. To do this we are rolling out an exciting new “Altheahood” pre-registration system. Similar to “fiberhoods”, this lets our organizers get their community excited about Althea and committed to participating before building out to a certain neighborhood. We are looking for prospective people to try out this new program. Get involved by contacting us at deborah@altheamesh.com. Now is a great time to reach out and start building an Althea network in your community.

Mobile data is the major driver of world internet adoption. The majority of the world's internet users have a mobile connection. So why isn't Althea focused on providing mobile internet?

The short answer is that home internet is useful even if you only build infrastructure to one location. A cell provider that only works in one place is a horrible cell provider, no matter how fast that location is.

Modern cell networks simply can't service the demands of the home internet market. Leaving a major industry with little to no competition and sky high prices.

Why is internet infrastructure so expensive anyway?

Since only about 3% of American households have access to fiber people assume it's just hideously expensive to build.

But that's not the case, The average fiber deployment pays for itself in only 5 years. Any hedge fund or pension fund would love to have that sort of return, if it where simply a matter of money we would be drowning in fiber.

The real cost of fiber and other in-the-ground telecommunications infrastructure is coordination- access to thousands of different pieces of public and private property.

As an ISP you have to coordinate with home owners, business owners, cities, towns and counties. The enormous amount of time spent organizing a new internet service provider means that you will be paying employees to organize your network for years before you start servicing your first customer.

Internet infrastructure is very profitable to build, but organizing the construction it is such a time and money sink that even the wealthiest companies in the world have to cut their losses. This strongly advantages incumbent ISPs, who can afford to pay hundreds of different people who's only job is to keep contacts with local governments, permitting agencies, and businesses.

Then a whole other level of people whose job it is to work with each individual homeowner to make the connection happen.

The cost of the actual fiber and the construction required to put it into the ground is very small compared to the standing army required to get all the paperwork in order.

What if we could make coordination free?

Althea reduces the coordination problem to only you and your neighbours. The larger organization of a network is handled automatically by the system and updated second by second.

Once a home has an Althea router it will always be on the lookout for a better connection and switch to it instantly. Before Althea, getting an entire neighbourhood onto your new fiber line might take months or years to organize.

The network owner would have to visit every home, and setup every homeowner on the new, better and cheaper, connection. With Althea that can happen within 30 seconds of the new line being plugged in.

By removing the need for a single organization to coordinate every part of the network and replacing it with an automated system what was previously nearly impossibly expensive to organize becomes free.

Building automatically efficient infrastructure

Next generation telecommunications networks are all about solving or reducing the coordination problem.

5g requires microtowers on every street corner and bets on reduced cost by reducing coordination with homeowners in exchange for increased city-level permitting.

Low earth orbit satellite internet, like StarLink and OneWeb, focuses on removing the need to coordinate with cities but still requires an antenna on the roof of every house.

Althea's premise is to tackle the coordination problem directly by changing the way networks are owned and built. Comcast's value is not the 30 pounds of copper wire and two servers it uses to service your home, but the coordination required to replace them. Althea provides an alternative to the monopoly owned network, a decentralized ISP owned by it's users.

• Dashboard overhaul!
• Local low balance warnings
• Automated failover at the exit level
• Phone registration for exits
• During periods of very low balance (below 10c) routers will throttle locally, preventing disruption to downstream clients and the tricky situation of entering and leaving the free tier every time the balance hits zero
• Temporary fix for peer discovery on particularly slow machines

This release mostly focused on improving exit resiliency. Althea now supports multihomed exits, meaning a single exit can be run across multiple datacenters and bridged into the mesh like any other alternate route.

The other major change is the creation of a 'low balance' mode. It's mostly a solution to the problem of a relay node running their balance to zero.

Because traffic is encrypted it's actually not possible for the upstream node to identify the non-paying relays traffic and only apply the free tier to that. Instead they must put everyone south of that relay onto the free tier.

Obviously this isn't good, so by having nodes voluntarily throttle when they reach 10c balance we can at least keep this from happening by accident.

What's coming next

Multi-homed exits where sort of an existential issue, as our networks grow downtime becomes even more critical to avoid, so they took priority over debugging some of the remaining underpayment issues.

Beta 4, if I have my way, will entirely focus on fixing two bugs.

1. Underpayment

As a pure percentage the number of users underpaying is now about 20%, down from around 50% before the last release. It's also dramatically less severe, with the average underpaying node only underpaying by 15%.

The remaining problems will require a more subtle and research-focused touch to really get this down to zero.

2. Lack of recovery

When the gateway or the exit goes down or is restarted for whatever reason there is about a 5% chance that a node will not reconnect properly.

I've been chasing this one for a long time, it's difficult to gather data on a bug when the bug is defined by a node not being online to tell you anything.

I think I've finally picked up the trail after debugging a strange issue brought on by remove the startup wait for Rita. I'm speculating that PeerListener hits some error code when sending it's multicast packet, doesn't handle it and drops the interface from it's watch list.

These are the sorts of issues you need a decent number of real users to encounter at all, so I'm glad we've gotten far enough to tackle fixing them.

As we pull up to the house, Shaun eyes a dense clump of trees between the house and our gateway antennas dubiously. “I bet we can't get good signal”, he says and attempts to avoid my disapproving glare by gulping his coffee. Not easily dissuaded, I have come prepared with maps and a link budget that shows we can set up a link here.  We usually start our planning by using google maps and Ubiquiti's link budget tool, and I have done my homework. We look at Ubiquiti's app on my phone and the line indicating aiming is fortunately located just to the left of the trees.

The homeowner spots us getting our “staff of aiming” out of the car and comes to greet us. The staff (+10 to knowledge) is a Ubiquiti litebeam attached to a sturdy pvc pipe, which we use to walk around a homeowner's property to find the best signal.  She's heard about our new type of internet service and likes the idea of being a part of a cooperative, supporting a local group of organizers, and getting great internet service. She's a real estate agent in the area, and knows how many homeowners are underserved in our area - left behind by corporate ISPs that took millions in federal tax dollars and then sold out to big national companies. We also talk about the importance of fast internet, and I tell her about the neighbor a block away that was so excited about his great speeds that he came running out of his house, laptop open in his hands, to show me the amazing improvement our new internet was.

I waved to my nephew across the street washing his car. We will install at his house the following week. He and his roommate are excited to be a relay to the other apartments in the complex. As subnet organizers, we'll help them keep things going well, his neighbors can call us if they have any problems, and he can make a few dollars by forwarding bandwidth along. Our Althea Subnet organization is set-up as a legal cooperative and we collect revenue from a set monthly fee, which creates a sustainable source of revenue as we support the diverse set of homeowners and business who have relay antennas broadcasting throughout the community.

Despite our best efforts not to forget anything this time, we have forgotten our socket set, so we welcome the help that her and her boyfriend offer. In fact, it's very common for us to arrive and receive a helping hand with mounting or even crawling under the house to connect the cable. I think people like helping - this isn't like getting a cable hookup and  waiting forever between the “hours of 12-4” knowing the installer will not arrive until the next day. It's a way to be a part of something, to build something meaningful and sustainable. One of the local business owners who also has an Althea relay, said “I like being a part of something bigger”. We were able to hookup a couple who hasn't had internet since the dial up days from her relay, which is definitely “something big”.

After mounting the antenna, Shaun and I joke about “talking to the aliens”, as what looks right for aiming the vertical axis seems a little different according to each of our perspectives. Thankfully, we can look at the radio dashboard on the laptop and slowly pan along the horizontal and vertical axis. A few db of signal strength can make a large impact on throughput, so we always take our time to find the best signal.

A quick check of the radio dashboard indicates that there are several relays nearby, and we choose the one with the best signal strength. In the future, we can set up another antenna and her Althea router will automatically route through the best and least expensive route. She can also become a a relay for her neighbors by using another antennas as an access point for other houses nearby. For now, we are just installing one antenna and Althea router.

With the radio mounted and aimed we carefully routed the ethernet cable along the siding and in to the home. I always like to make sure we are clipping the ends off of the zip ties, and that service and drain loops are tidy. While Shaun terminates the end of the ethernet cable, I tell her about how she can load her router up with tokens as she needs. I help her install the Coinbase app on her phone and show her how to scan the QR code in her router dashboard to send ethereum to her router. It doesn't take long, about 10 minutes, and she has the hang of it. All she has to do is load up the router with tokens and her router will do the job of sending micro transaction payments to any of the other relays along her route, and to the subnet organizers, for their monthly fee.

As we finish up and begin packing up our tools, I spot another one of her neighbor's antennas and smile. It's a manifestation of Althea's decentralized system. A variety of different entities have ownership and are participating in seamlessly peering and reselling bandwidth across the network. If the dream of decentralized network ownership can happen in rural Oregon, maybe it can happen anywhere.

In retrospect why is pretty obvious, we released payments and then immediately set about doubling the size of our test network. In doing so we had raised the bar for the quality of our features and interfaces.

Many of the quirks that were previously acceptable were now major annoyances. So Beta 2 became a four week long marathon of much needed user experience polish.

What's new in Beta 2

• xdai support that actually works well
• Fixes for various UI inconsistencies
• Improvements to peer discovery
• Fixes to port toggling
• Much faster startup
• Better detection of crashes and automated recovery
• Improved firewall setup
• Fixes for stuck nonce preventing payments
• Performance optimizations for accounting code

Payments roll-out

We are now at 100% of the network running Beta 2 and loaded with Ethereum to pay for their access.

The anti-overpay protections seem to be working flawlessly as all of the billing issues we've encountered have involved nodes underpaying. The current breakdown is about half of the network paying without any issue and the other half underpaying in some way.

Of those underpaying, a minority were underpaying dramatically due to a bug that caused their nonce values to desync from the blockchain, preventing new payments from being sent. This is fixed in Beta 2.

The rest are underpaying by a value between 5-10% which we've attributed to packet loss from radio links. This is only really a problem for exit -> client traffic since that's the only pay to receive scenario. In the general case packet loss will only result in overpayment since it's pay per forward.  

The exit-client relationship is already fairly trustful so we can just couple what the user pays to what the exit sends and not necessarily what the user receives. With the usual safeguards against wallet draining attacks.

Intermediary < Relay

We now have 3 relays, meaning users who are also hooking up other users.

We used to use 'intermediary' as the term for this but in the process of actually getting people started we've found that relay is more quickly understood by most people so we're switching terminology.

We also had our first completely organic install in the Clatskanie network, a relay connected and setup their user without any intervention on our part.

The friction points for that case make up about half of our beta 2 fixes. While the relay had no issues with the actual physical setup several parts of our software where mislabled or confusing.

Upcoming features

With payments out and pretty rapid expansion on the table we're mostly focusing on user facing improvements for the time being. Like this new router dashboard.

The network DAO features continue to lag behind where I would like, but with payments online I expect to have it out the door within the next two release at least. Depending on how well dialling in bandwidth payments goes.

It’s difficult for big telecom to make money selling bandwidth to small rural towns or low income urban areas, otherwise they would already be there. Internet poverty exists because there is little economic incentive for those who build the internet infrastructure to provide service to these areas. This is further complicated by the centralized nature of legacy ISPs, which are slow-moving and utilize rigid and expensive pathways to build out infrastructure.

Currently, government granting programs funnel billions of taxpayer dollars to big telecom in a short sighted approach to bridge this digital divide. While this has led to some infrastructure improvements, the recurring revenue per user in these areas is small enough that many of the big telecom providers build very minimally and are unwilling to invest in ongoing upgrades. There are still 24 million Americans without access to broadband internet.

Several blockchain Internet projects like RightMesh, Ammbrr, and my own project, Althea, are sustainably solving this problem by incentivizing people to build their own wireless internet infrastructure for their communities. By combining the user and the operator as one entity, the incentives are aligned – the interests are the same. Community members can be encouraged to setup and maintain their own networks if incentives are available for participation in a decentralized ISP.

Hardware investment and use of existing rooftop space is an important part of any wireless network. These new models look to the example of Uber or AirBnB to imagine how people could use their homes and businesses to make money by leveraging resources they already have.

Decentralized systems, which are agile, resilient, and quick moving, are more fit to provide service in many areas. It is the inherent administrative inefficiency of big telecom companies which causes excessive spending and slow implementation. Business people in offices miles away will not have an understanding of the community or how to most efficiently build a network there. Residents of a local community have a deep knowledge of the landscape and connections to property owners that big telecom will never be able to match.

In a decentralized ISP, each home or business can connect to each other, earning money with their property while competing in a decentralized bandwidth marketplace to best serve their neighbors. Community members and small business owners are motivated to create new links to nearby neighbors because they earn money for their participation and hardware. These protocols also incentivize redundant links, adding resilience, and driving prices downwards.

Healthy, thriving networks are being made possible in places that centralized telecom has failed.

*Originally published in BlockTribune December 2018

Rolled out @AltheaOrg Beta 1 today in production. Members of the Clatskanie Co-op are loading up their routers with ethereum, which automatically pays my router for bandwidth. Exciting stuff! pic.twitter.com/uCUftWoDDG

— Deborah Simpier (@DeborahSimpier) January 23, 2019

The last couple of weeks have been busy, we're at about 50% Beta 1 rollout in our test network.

Converting users has been somewhat time consuming since most need the 10 minute crypto primer. We could have probably tried to automate that but we decided that a high touch process would let us find user annoyances and problems faster.

we have our first example of an intermediary, earning ethereum ($ETH) by forwarding bandwidth to a neighbour in his apartment. In this case, it cost less than $100 to set up his neighbour with high speed internet and both he, the gateway, and the subnet organizers will make recurring revenue.  Several larger intermediary nodes serving between 10 and 50 households will be rolling out in the coming months.

Bug Fixes for Beta 2

While Beta 1 had Xdai as a selectable payment currency it wasn't actually functional. Do to some quirks of how the POA public RPC is setup and our own Rust libraries we can't easily query it.

Beta 2 will include a set of fixes that use our own xdai RPC cluster as well as some user convenience features that prevent two routers form accidentally trying to pay each other in different currencies.

Our South American test network has been insistent on a stablecoin as a medium of exchange so it's been a priority to get xDai working.

Fingers crossed on a rollout next week down there.

Other improvements include drastically reduced overhead for those devices with remote logging enabled. Devices in our test networks are actively monitored for bugs, in Beta 2 we're introducing log compression, which reduces the upload overhead from this by a factor of nearly 200.

UI improvements

Now that real-money operations have started we've been provided some great opportunities to do user studies on our various interfaces.

One of the larger goals with Althea is to make building a network seem as approachable as a home improvement project, rather than an arcane combination of incomprehensible systems.

With that in mind exactly how we present concepts to users is one of the most important things for us to really nail down with our test deployments before trying to address a wider audience.

New blog platform

If you haven't already noticed we finally got around to deploying our self hosted blog rather than using Medium. We originally went with Medium hoping for some organic views and to get away form our markdown powered static site.

But ultimately Ghost proved to be the better option to allow non-technical users to write posts without sacrificing control in exchange for a relatively small number of Medium referred views.

Ipv6 Range

As of today we have acquired 2605:1C40::/32 for assignment to Althea routers.

We plan to peer this network using BGP and seamlessly allow all nodes within an Althea network to be fully integrated into the global internet and accessible as a first class citizen of the Ipv6 internet.

Facing Forward

While payment channels have made good technical progress their roll out is on hold

Transaction fees on the Ethereum network have been even lower than expected, resulting in payments every 8c to 12c when paying 5% in fees.

At that level we really can't justify focusing on channels when any user experience or other bugs remain.

Our plans for the immediate future are focused on going to market and really nailing down a playbook for quickly starting and expanding networks.

This is our first major (longer than a 15 minute) outage since June of 2018. The technical issue exposed is pretty minor, the real feedback here is about a series of failures in our incident recovery procedure.

Primary Causes:

• Incorrect configuration of per process file descriptor limits on the exits
• Recovery failure of some client devices

Failures in incident response procedure:

• Lack of validation of on call procedure
• Lack of secondary emergency recovery contacts
• Incorrect configuration of the service autorecovery on the exits

If any one of these issues had been resolved beforehand this would have been limited to a minor outage.

Primary Causes

Incorrect configuration of per process file descriptor limits on the exits

Getting into the actual technical issue at the root of this. Exit servers forward connections from clients in the Althea network.

Each of these connections takes up what’s called a ‘file descriptor’. While the system wide number of file descriptors had been properly tuned to prevent issues the per-process limit of file descriptors was limited to the default 4096.

Following a momentary networking issue, either within the Clatskine test network or on the larger internet there was a rush of traffic and requests to the Rita exit process on the exit.

Since Rita exit must claim a file descriptor for each connection it has it exceeded 4096 during this period and crashed.

Jan 14 22:55:39 localhost.localdomain rita_exit[26857]: [2019–01–15T03:55:39Z INFO rita_exit::rita_common::network_endpoints] Got Payment from Some(“[fddf:bb15:dd3e:ebf4:2524:550:bb38:50c2]:56648”) for 6360000000 with txid None
Jan 14 22:55:39 localhost.localdomain rita_exit[26857]: thread ‘main’ panicked at ‘called `Result::unwrap()` on an `Err` value: Os { code: 24, kind: Other, message: “No file descriptors available” }’, src/libcore/result.rs:1009:5
Jan 14 22:55:39 localhost.localdomain rita_exit[26857]: note: Run with `RUST_BACKTRACE=1` environment variable to display a backtrace.
Jan 14 22:55:39 localhost.localdomain rita_exit[26857]: Panic in Arbiter thread, shutting down system.
Jan 14 22:55:40 localhost.localdomain systemd[1]: rita-exit.service: main process exited, code=exited, status=101/n/a

As a major network service with potentially hundreds of clients Rita exit should have both been configured to deal with this many connections and have code to prevent too many connections from occurring at the same time in the first place.

That being said if the incident response team was on point this would have been a 5 second fix to avoid downtime.

service rita-exit restart

Recovery failure of some client devices

The Clatskine gateway was also taken down by this incident and failed to automatically recover itself.

This can be attributed to the watchdog script checking only the liveness of Rita and not if Babel actually has any routes.

It will be revised to more carefully check that Babel is functioning correctly and not simply limping along in a zombie state.

This was the most non-trivial failure for this incident, as it had to be recovered on site.

Remote access will be setup for the maintainer of the gateway to make this process easier in addition to better automated recovery.

Failures in incident response

Moving onto issues with the recovery response. Recovery for this incident should have ideally taken less than 15 minutes and as few as 5 minutes. Instead it took over 6 hours.

Lack of validation of on call procedure

Late last year the Android phone of the primary on call contact received an update to Android Oreo.

During this update the Do-No-Disturb feature had it’s behavior changed and configuration reset.

Instead of defaulting to a higher ringer volume for priority calls the call was allowed through but using the system ringer volume, resulting in a vibrating phone that did not wake the on call contact.

Lack of secondary emergency recovery contacts

Emergency recovery documentation was in place and accessible. But the technical personnel responding to the issue did not have ssh access to the primary exit server.

A ssh key was requested at install time, but not followed up on. Resulting in only two people having access to the production system at the time of the incident.

Of these two only one was on call and as previously described the on call procedure failed.

Incorrect configuration of the service autorecovery on the exits

Due to the transient nature of this issue simply restarting the Rita exit service on the exit would have sufficed for restoring the system to functionality.

Anyone with access to the system could have done this trivially. The service itself could have also been configured to attempt a reboot upon failure.

By adding these two lines to the service template the exit would have been automatically recovered.

All services are now back online, we hope to have all issues in this post mitigated by the end of this week.

Once this release passes QA in a couple of days we will have out first Althea build with feature complete routing and bandwidth payment.

Beta 1 will be able to, dynamically route, buy , and sell bandwidth with production Ethereum. Everything described in our original demo video, but ready for use as the primary ISP for the households and businesses of our test networks.

There are lots of features left to add, including proper integration for our Network DAO organizations and of course as the first Beta release there’s a non-zero chance of money eating bugs.

Why real Ethereum?

Our last release Alpha 12 was our first and now our only testnet release.

The reason we’ve chosen to push forward with using real money so quickly is a combination of very good testing results from Alpha 12 and the high cost of downtime.

People really, really, don’t like it when their internet goes down, the cost in terms of customer dissatisfaction that comes with any service disruption is much higher than a month’s worth of internet service money.

Because of that there’s not much point in cutting over our production networks to builds that both enforce payments and still use a testnet. Once the possibility of reduction or disruption of service is there due to a major network change it’s cheaper to just compensate for lost Ethereum in our test networks than it is to risk downtime doing this all again after testing Beta 1 with testnet Eth.

What’s New in Beta 1

• Real eth payments
• New traffic flow optimizations to improve performance
• Free tier & payment enforcement
• Configurable maximum price
• Billing code optimization
• Dynamic payment adjustment, as transaction fees change the payment frequency is adjusted to keep fees at 5% of total costs
• A configurable pricing oracle to compensate for swings in Eth price

What’s not yet in Beta 1

• Updated subnet dao integration
• Querying various network wide values from the subnet dao
• Advanced fraud detection (subtle route degradation)
• Payment channels

Production Network DAO

In news that would usually be important enough for it’s own update we have our Aragon Althea network app deployed to mainnet where it is being used as a in-production list of nodes.

Votes of the network organizers are working as well, the missing major feature is the ability to collect network fees natively. Right now they just have to be sent to the DAO manually.

What’s Next?

In 2019 we are setting out to demonstrate not only that normal people can build an ISP but also that decentralized cooperative networks are the superior way to do so.

Once we’ve operated Beta 1 in our test networks for a week or so I plan to cut Beta 2 with whatever bugfixes we find necessary and start recommending it for general usage by those following our project.

The Network DAO Aragon organization has also made significant progress it’s available on Rinkeby here. Feel free to play around with it.

Our next release is going to be the big one. Beta 1, which is by definition feature complete and ready to be used (cautiously) with real money in the real world.

Implementing the free tier

The last major feature we need to implement to reach Beta 1 is enforcement. What you actually do if people don’t pay.

Althea doesn’t ever actually cut anyone off, a free tier with limited bandwidth is provided gratis to help users get setup and generally use the internet. All the routers in the network agree not to bill each other for the free tier level of traffic.

The exact speed of the free tier is decided by the local network organization.

Having the free tier be enforced with a speed limit at all is a little anachronistic for Althea. Like I explained in our billing blog post connection speeds are hardly reflective of internet usage experience.

But it’s much easier to implement a speed limit than it is to make layers of prioritization based on bidding.

The life of a packet in an Althea network

I’ll expand on each part of this diagram in later blog posts. But to focus in on what we’re changing in Beta 1 lets talk traffic queuing.

In the Alpha 12 diagram the only queue operation we have is fq_codel, which is a traffic prioritization algorithm designed to try and reduce latency and make everyone’s connection ‘feel fast’.

If you have any interest in the plumbing of the internet I recommend this talk from around the time Codel was first being designed. It’s an excelent source of background, intuition, and tooling.

Bufferbloat: Dark Buffers in the Internet

As powerful as Codel-derived tools have become since that talk a careful reading of the documentation will make it obvious that a few suboptimal decisions where made in this first network design.

The codel operator on the router side doesn’t actually serve any useful purpose as it’s can’t identify connections because of the encryption.

Likewise the codel queue operation on the exit side isn’t ideally placed. While this one can at least see the traffic it’s supposed to be managing that management is performed over every connection coming in and out of the exit.

This can lead to globally optimal decisions but is less sensitive to a single routers problems.

Our new design for Beta 1 moves the codel operators so that they have a better view of traffic to manage and are generally closer to the ‘edge’ of the network.

With one placed in front of the router LAN and the other placed after mesh routing in order to prevent a single router from hogging traffic across the network.

I’ll be keeping a close eye on the new codel layout and hopefully find ineffective operations a little faster this time.

The other addition to our queue system is HTB (Hierarchical token bucket, no relation to crypto tokens), which is a pretty old fashioned bandwidth allocator. Using HTB we can reduce peers to the free tier level of bandwidth.

Sadly HTB isn’t really flexible enough to operate across interfaces. Meaning we can’t pool multiple interfaces worth of free tier bandwidth in such a way that also makes sense from a billing perspective.

Like I said before the use of a speed in the free tier system at all is not ideal, but it will do until we can implement a more elegant way of accounting for ‘unsold’ traffic.

Traditional ISP billing

This is a pretty typical advertisement from an ISP. Just looking at it it’s hard to tell what you’re buying. What does ‘up to’ mean in the fine print?

ISPs universally use a tactic called ‘oversubscription’ where any given amount of speed is sold to many people at once.

For the most part this allows them to inflate their speed numbers harmlessly. Most people aren’t using their internet 24/7 and will never notice they are sharing the speed they buy.

The devil is of course in the details.

Let’s say for example that you’re a traditional ISP, you hook up a neighborhood with 100 homes using a single 1gbps connection, that’s 10mbps per person. Twenty is considered a pretty safe oversubscription ratio, so you sell each home a 200mbps connection.

Obviously not everyone can have 200mbps at once; but a HD Netflix stream is only about 6Mbps. So a family of four each streaming their own show will barely dent the capacity. Even one or two gamers downloading during peak hours won’t be enough to noticeably cause any problems.

For the most part these users will ‘get what they pay for’ in the sense that a speedtest will read 200Mbps, but what they pay for is very different from what they use.

Oversubscription breaks down when it becomes likely that a normal household will actually use the entire connection they are being sold when any regularity. For example a 20Mbps DSL connection will easily be maxed out by that same family of four we were talking about earlier.

The statistics collected by the FCC in 2016 bear this out.

Chart 15.3 from the FCC 2016 broadband report

American cable and fiber internet users get somewhere between ~80% of their advertised download speeds during peak hours. The number for DSL is much lower, with a mere 50% of the users getting the advertised speed during peak hours (Weekdays 6:30pm — 11pm).

Oversubscription isn’t inherently evil. But it decouples what ISPs sell from what people use and even what the ISP is actually capable of delivering.

This sort of intentional obfuscation of what you are buying and what is being sold is typical of monopolies. The more clearly defined the price and service, the more easily customers can replace one provider with another.

Althea’s billing model

Althea works like your electricity or water bills, you pay for what you actually use.

With Althea your home WiFi router acts as a personal shopper to buy internet bandwidth over the cheapest and best path every second, seamlessly. Even if you have only one connection yourself, the router will still get faster service and cheaper prices as the network reorganizes further upstream.

If you participate in passing traffic for other people you’ll find some of the money they paid as extra money added to your account.

It’s important to focus on the ‘personal shopper’ concept. You put money into your account to be spent over the course of the month.

Like a prepaid phone plan, it’s impossible to end up paying more than you have deposited into the account. Unlike a prepaid phone plan the money doesn’t leave your home until it’s used, it lives in your Althea router.

With a traditional ISP you must trust the privacy of your web browsing to the provider of the connection to your home. In most countries major ISPs sell your browsing data to advertisers.

Althea uses an integrated VPN to easily and automatically protect your privacy from whomever may be carrying your traffic. While you do still have to trust the VPN provider the selection of who to trust with your browsing history is no longer limited based on your location.

How does it work out for me?

The biggest question that new users give in our payment experiments is, “how much data do I use?”. I’m going to try and put internet usage into context using my personal internet usage as an example.

Right now I pay $70 a month for gigabit fiber to my apartment. In the grand scheme of American Internet access this puts me in the top ~5% of households.

We’ll be comparing my current internet to an Althea network charging 15 cents/GB

I’m a moderate internet user. I game, stream media exclusively for entertainment, and partake in the strange hobby of downloading Wikipedia regularly.

On average this comes out to about ~370GB a month in usage for me.

Let’s look at where my 15 cents / gigabyte to the Althea network would go.

10c goes to the gateway, which is purchasing a resalable fiber connection. (about $1000/mo)

4c goes to my neighbor, Alice, who is helping get the connection from the gateway to me.

1c goes to the exit, who is peering traffic out to the internet.

During an average month my I would spend $56 on internet.

$37 of my bill goes to the gateway

$15 goes to Alice

$4 to the exit

Based on these numbers I would have saved $174 this year using Althea.

I chose to use myself as an example here because a lot of our potential users are concerned about exactly how much bandwidth they use.

Hopefully by providing a frame of reference for a pretty serious internet user we can put some of those fears to rest.

Ultimately pricing is up to the market

Our test network in Clatskanie is charging 20c a gigabyte, this is a non-trivially worse price than most of what the United States pays. But since most of the town is on 5mbps DSL, the network is adding a few new people every week anyway.

The market there does not demand lower prices, but instead acceptable broadband service first and price second. The prices have been set by the various equipment owners accordingly.

As the network there matures, I’m sure competitors will enter and prices will be pressed down, but ultimately Althea achieves lower prices by making it easier to compete, rather than through some inherent property of the billing system.

In our recent updates I’ve been talking about our metered billing tests and careful work implementing the client-side components of our payment channels.

For those that don’t already know payment channels are a blockchain application that allows two parties to create a pool of funds on the blockchain that can then be freely swapped offline. This is the foundational technology that will allow your Althea router to pay it’s neighbor fractions of a cent at a time.

Sadly payment channels are fairly complicated to setup and it’s clear that working out the logic is going to take a several more weeks. In frustration we looked up how much it would cost if we just sent the money the old fashioned way to the blockchain.

Surprisingly this was a lot more feasible than the last time we checked. Due to the current price of Ethereum it’s viable to send a transaction every 36 cents of usage and pay a 5% fee. That comes out to about once every hour.

This is nearly free compared to credit cards, which usually start at 20c or more as their minimum fee. More than good enough for us to work with for now.

Once that was decided it only took a few days to get my testbench routers to start paying for their bandwidth.

This lets us get onto testing the rest of the system in parallel. Accelerating our timeline by at least a month.

Limitations on channel-less payments

Obviously if the whole world used Althea this wouldn’t be viable, likewise it’s always possible for CryptoKitties to strike and lock up the entire blockchain again.

To remedy this temporarily I’ve added some dynamic fee computation that will keep the fees to 5% by slowing down the payment rate as the network grows more congested.

If fees increase to 10x their current amount routers will adjust to send payments for every $3.60 of bandwidth used and so on.

At a 100x increase in transaction fees this strategy is no longer viable, paying every $36 of bandwidth used simply exposes the user to too much fraud risk.

The risk of fees increasing more than 10x in the time period between now and when channels are ready is pretty small.

What’s left?

Network DAO payments:

We’re doing some final patching on the Network DAO app. I wouldn’t be surprised if we had our first Network DAO up and accepting real money by my next update.

Bandwidth Payments

With channel-less payments I estimate QA will take all of this coming week and into next week. Followed by deploying testnet payments to our Clatskanie network.

Provided that goes well we just need to finish up the free tier and enforcement. These are pretty straightforward features, hopefully we’ll be moving real money no later than the New Year.

This update is coming a little late; it’s been a travel filled couple of weeks.

We held Altheapalooza, a small conference in Portland featuring the Althea team and Graham from startyourownisp.com.

I also presented at the Nashville Blockchain Developers meetup

The goal of Altheapalooza was to create video content about every detail of setting up an Althea network, from antenna selection to spectrum planning. This is also our first experience trying to teach all of this in a more formal setting rather than ad-hoc lessons in the field.

This was a great opportunity to get the team together, visit the network in Clatskanie and start to tackle one of the fundamental challenges of setting up an Althea network, can we effectively train people to setup real world wireless gear?

Where on the sliding scale of a two day in person course all the way down to a 15 minute video clip do we need to be?

Althea can automate every other part of the process, but point to point wireless gear is the only way to provide incumbent ISP competitive (and often superior) performance at a low cost. We need to be very effective at teaching people to use it.

We’ll let you know how this experiment progresses and once the videos of Altheapalooza are ready.

Update on bandwidth payment experiments

As expected once we started paying close attention to bandwidth payment amounts we found a few bugs. Now fixed in Alpha 11, rolling out over the next few days.

What’s New in Alpha 11

• Separate default WiFi names to prevent issues with client devices being dumb about frequency selection
• New logarithm based route selection weighting prevents realistic prices (hundreds of thousands of Wei per byte) from causing routing issues
• Tools in Rita for price setting and quality adjustment, updated to work with more realistic price values
• Fixes to client and exit billing to properly account for nat traffic
• Crash watchdog for Rita should restart Rita on crash automatically
• more robust automated update script to handle some common failures we’ve seen

Technical progress update

Network DAO Payments

As you may have noticed Aragon is live on mainnet, this changes our previous timeline. Which involved figuring out how to deploy Aragon ourselves.

Right now we’re working on pretty minor integration issues, trying to get our Network DAO app and it’s assets to show up on the Aragon web interface. We have contacts on the Aragon team helping us out.

1. Aragon deployed on mainnet
2. Possible to install the Althea app into an official Aragon DAO
3. Althea Network DAO app shows assets and is generally usable < =
4. Billing integration with the rest of the router functions, to provide more seamless payment.
5. Create light client that works on routers and embedded devices.
6. Create a bidirectional payment channel and deploy it.
7. Integrate the light client and existing billing code.
8. Integrate the #3 with the payment channel.
9. Spec out channel flow in detail < = forgot to do this
10. Integrate billing code with existing accounting code. < = back here now
11. Comprehensively test this entire stack.

We’re approaching testnet readiness with bandwidth payments and we’ve started working on how to present the concept of device to device payments to people in the real world.

This is an exciting milestone that brings up some very important questions.

• What kinds of incentives are really best at motivating people to sell bandwidth?
• What kind of investment are people willing to make to do so?
• What pain-points can we address in the design?

Designing an experiment in payments

Testnet bandwidth payments are on schedule to start next week, but I won’t be surprised if it’s the end of November before they are stable enough for real world testing to begin. With truly autonomous payments there’s a lot of edge cases with wallet and channel management that you have to answer correctly.

Ideally we want to work on figuring out how to present and design bandwidth payments from a human perspective in parallel to the technical completion.

So we came up with a quick and dirty idea that lets us perform a small scale experiment in our of our test networks.

The payment channel transactions aren’t quite ready but we do perform bandwidth accounting correctly. So instead of turning that into a transaction and sending it to the blockchain we send it to a server. From there we can pay users in our Clatskanie test network for bandwidth by hand.

It’s not exactly elegant and doesn’t really work well at all unless you have people on the ground. But it does allow people to be confident they will be paid for expanding the network and make money installing nodes.

Which is all we really need to observe what motivates people and iterate on how we present bandwidth payments and Althea to real people as a product to use. Not just a technical curiosity.

Some interesting lessons learned already, as we prepare to roll this out next week.

• People are more motivated by saving $40/mo than making $40/mo. Hooking up a single neighbor for the money is a less powerful motivator than ‘being a good neighbor’ and helping them save money.
• Somewhat hilariously nearly everyone wants to help out their neighbor but almost no one totally trusts their neighbor to help them out.
• Money does become a clear motivator once you reach ‘side hustle’ status of at least a couple hundred dollars a month, so hooking up several homes.
• Cheaper internet is much less of a motivator than bad internet when it comes to people being interested and willing to try Althea.

Technical Progress Update

Network DAO payments

1. Deploy Aragon OS
2. Deploy Aragon OS apps
3. Deploy Network DAO factory < = we are here
4. Network DAO factory can be called to deploy a new Aragon DAO with the Althea DAO app preinstalled.
5. Create light client that works on routers and embedded devices.
6. Create a bidirectional payment channel and deploy it.
7. Integrate the light client and existing billing code.
8. Integrate the #3 with the payment channel.
9. Integrate billing code with existing accounting code. < = we are here
10. Comprehensively test this entire stack.

This is the vision which is front and center in our materials and which has inspired people all over the world to start planning and building Althea networks. The ideal is that if everyone in an area is incentivized to strengthen the network, then they can grow the network without the centralized coordination of a traditional ISP.

Deborah has been helping people plan these networks, alongside building her own Althea network in Clatskanie. As these networks are moving past the planning stage and into reality, we’ve been hitting up against some of the hard facts of the ISP market. One of the main points of friction that the basic Althea system has with the way business is done today, is that wholesale, resalable internet access usually involves very large connections with high monthly costs, and long minimum contracts.

For instance, a 500mbps dedicated connection in Clatskanie costs $1400 per month on a minimum 3 year term. This is not a huge impediment in starting a traditional ISP, you just plug it into a spreadsheet along with the rest of your startup costs, monthly charge, and expected user base, and either the numbers add up or they don’t.

But in the basic Althea model, we ideally want there to be different “gateways” (people paying for this large subscription and selling it into their local network), who coexist and compete to some extent. But that’s tough to bank on. While Althea’s per-gigabyte bandwidth payments promote viral growth by allowing the system to reward nodes granularly, they are also inherently somewhat uncertain. Having fixed payments can be beneficial.

That’s exactly why we designed the Althea governance system the way we did. The governance system works alongside the bandwidth payment and routing system, but it involves each node making fixed monthly payments to a “subnet DAO”, an on-blockchain organization run by a group of local people who are promoting the network and providing customer support. Because each node on a network is identified by an IP address, these are known as “IP leasing fees”.

These subnet DAOs are meant to be separate from the gateway nodes which are connecting to the internet and the intermediary nodes who are forwarding traffic through the network. In a large, mature Althea network, we want many different players to be able to coexist and coordinate more efficiently than they would under the roof of one monolithic firm. For example, multiple subnet DAOs could provide multiple levels of service while utilizing many of the same gateways and intermediaries in an area.

However, it’s becoming clear that in the early days, the people in a subnet DAO are often the ones coordinating a build-out of the network at all and are often taking on the cost of gateway bandwidth. We are going to start recommending a higher subnet DAO fee (30–50% instead of 5–10%), and as we guide people in starting their own networks, having them plan them more along the lines of a traditional ISP, until the network is large enough to sustain internal collaboration and competition of different independent players.

The Althea software works great in this mode as well. This doesn’t actually involve any changes to the software at all. Justin Kilpatrick and the rest of our dev team have done a really great job writing software that helps people easily set up networks without many of the configuration and equipment selection headaches that usually come along with a new network. Althea networks configure themselves autonomously (barring a few stubborn bugs!). Network architecture is not something that the person setting up the network needs to think about with Althea.

As Althea networks mature, and people using the network understand that they can earn money reselling, strengthening, and expanding the network, bandwidth payments will become a bigger part of the picture.

As I talked about in the last update we’re trying to move quickly on payments by getting the network DAO payments onto prodnet (real money) and bandwidth payments on to testnet (fake money) this month.

Both of these are somewhat aggressive but possible.

Network DAO payments

1. Deploy Aragon OS
2. Deploy Aragon OS apps
3. Deploy Network DAO factory < = we are here
4. Network DAO factory can be called to deploy a new Aragon DAO with the Althea DAO app preinstalled.
5. Create light client that works on routers and embedded devices.
6. Create a bidirectional payment channel and deploy it.
7. Integrate the light client and existing billing code.
8. Integrate the #3 with the payment channel. < = we are here
9. Integrate billing code with existing accounting code.
10. Comprehensively test this entire stack.

What’s new in Alpha 10

• Port toggling: The ability to switch port functions dynamically, allowing users to customize their device functions more easily.
• Interface internationalization, you can now select Spanish for the Althea router interface!
• Remote logging, there’s now an endpoint users can hit to send us router logs if they encounter an error in the field. We hope to put a button on the dashboard for this in the next release. It is of course totally disabled by default.
• Bugfixes: very specific billing edge-cases can no longer crash routers

Thoughts on routers and pricing

As of about a week ago it’s possible to easily buy preflashed routers and other networking equipment from us. There is also a hopefully complete set of documentation on how to connect to other nodes and setup your router.

You’ll probably notice we’re not selling the cheapest option on our supported routers lineup. You can generally go as low as $20 for a working device, but we’ve found that ~$100 is where you stop getting a better router for your money.

After testing dozens of routers most of the $150 — $300 routers you find at Best Buy are generally no better than the $100 option if you know what to look for.

There’s a reason why the Best Buy options are so expensive of course, having a higher priced options attracts people to the ‘middle’ of the price curve. Those shiny $300 routers are designed to justify in your mind the $150-$200 routers.

We experience this same issue putting up a $100 router next to a $25 one, except in our case we’ve made sure the $100 router is really 4x better (it’s actually 5x better according to my testing).

I suppose we could throw in a $300 router that’s not actually 3x better than a $100 router. But I find that hard to justify, so for the time being we’re going to try and steer first world buyers to the GL B1300.

We created Clarity a minimalist ultra-lightweight Ethereum transaction generation library that can run on any device powerful enough to reasonably sign translations.

In parallel to bandwidth payments our local network DAO’s should be ready to collect payments very soon. Since we are using Aragon as a framework for storing the money it’s mostly audited already. Which should dramatically reduce our time to payments with real money.

Our latest release contains only minor networking fixes, at this point networking is stable enough that we will need an order of magnitude more users to really start running into new bugs.

Two of our own will be traveling to Colombia next week to help setup our first South American deployment.

Short term roadmap

Our goals right now are essentially payments and documentation ASAP.

We have new networks which will quadruple our number of active users in the next few weeks but without some form of real money billing these are all experiments. Not decentralized businesses.

We’re focusing on getting Network DAO payments online and integrated with Aragon. Not only is this simpler than bandwidth payments technically since the heavy lifting has already been done for us. But it’s also better suited to bootstrapping a network. Where bandwidth payments may start very small anyways.

The other side of this problem is that unless you know how to poke around github repos or talk to us how to get started with Althea is a difficult thing to figure out. Like I said in my last update we’re really riding on our website refresh to achieve that.

Why Clarity?

After significant research we found that the Parity Ethereum client is simply inappropriate for embedded environments. This left us in the awkward position of having to furnish our own lightweight transaction generation library.

After a detour of a couple of weeks Clarity is capable sending functional transactions from a variety of environments. Most importantly for us is support for the MIPS CPU architecture which is very popular in routers for it’s ease of manufacture and low cost.

There still needs to be some work done on documentation and sending transactions to contracts specifically, but I’m very happy with the progress.

Dual purpose WiFi cards

Althea is transmission medium agnostic, meaning no matter what connections it is provided it will make the best use of them possible.

A connection that’s always been less useful than people seem to expect is meshing using internal omnidirectional radios. In general Althea deployments focus on high performance point to point antennas and leave the internal router radios to normal ‘home router’ tasks. Mainly home wifi.

It’s possible to use a internal wifi card for meshing with AdHoc mode but only if you give up on using it for your own Home wifi. Giving up either home radio sucks, routers obviously provide both frequencies because both are useful.

Earlier this week I decided to investigate MeshPoint (802.11s) a WiFi standard that attempts to codify those ‘whole home wifi’ setups for connection sharing by creating a group network that many access points can join and connect over.

This is of course perfect for Althea, which simply sees yet another connection to make use of.

The real surprise is that careful inspection of the driver capabilities using the iw tool reveal that with 802.11s we can have our cake and eat it to. By sharing the same device for multiple access points we can provide connectivity without sacrificing user radio functionality. At least on devices that have up to date WiFi hardware.

It remains to be seen if the contention of running two networks on a single WiFi card causes problems.

Several new networks getting started (about 5) some made videos of their setups one, two.

With network features now generally stable a lot of people are getting online and using Althea. Likewise development has shifted gears from putting out fires to cranking out actual blockchain/payment features.

In parallel we are working on documentation and additional polish to provide easier access for people who want to try Althea now that it is approaching stability.

What’s New in Alpha 9

• Local network enforcement with early DAO contract
• Checks to wifi password and name to prevent accidental breakage
• Added api to view neighbours debts
• Update lots of dependencies
• Cleanup of old tunnels
• Dashboard interface refresh
• Rasbperry PI targets!

What we have right now is a pretty barebones selection screen and code on the routers that will query neighbors for membership on the local Network DAO once you turn it on.

We’re hoping to have payments and proper voting in the next few weeks.

Raspberry PI targets

As our most requested feature we now have Raspberry PI OpenWRT targets available on the releases and firmware pages. For lack of extra ports the Ethernet jack is a meshing port and the wifi is used for LAN connections.

It’s not really useful for real world scenarios, there aren’t enough ports to resell connectivity easily and still be able to use it as a home device. But generally everyone has one lying around to play with so it’s an easy way to try things out.

Documentation

We’re hard at work on a refreshed website and updated documentation. Our landing page has become painfully out of date and now that the software is ready for people to seriously use it the difficulty of navigating and understanding what we’re trying to do with our current messaging is making itself more and more known.

I’m really happy with the designs we have so far and I think you will be too.

Conclusion

In general I’m very impressed with the quality and speed of our progress since onboarding our new team members. At this rate we will easily achieve our goals before the end of the year and really get to see if Althea can scale to become a serious competitor to existing service providers.

I’m very happy to say that since our last update we’ve had about 8 devices be put into real world use without any of the failures/crashes that plagued the previous releases.

This is the first release to include all of the long term changes inspired by our recent push for stability. We wanted to see how it held up before really pushing it out.

We’re going to take this as a green light to start expanding into our wait-lists and getting people online.

You can find a guide to flash your own device router on our forum. We’ve also moved a lot of our development discussion there so you may want to peruse it anyway.

Our first blockchain-backed features going into testing next week. It’s the first part of our governance model, the local Network DAO that allows organizers to collect a small fee for fixed cost services (eg tech support) and allocate funds to network wide projects.

I hope to have a release candidate with that by this coming Friday.

The goal is to have this stabilized sooner than actual bandwidth payments, the network integration is very simple and the heavy lifting on the blockchain side is being handled by Aragon. This should allow us to have a way to collect real money while the more complex payments system is audited.

There’s a big motivation here to get real features out the door, we’ve been cultivating real world Althea networks for a long time now and shepherding them through the setup process.

Things like finding backhaul, hardware, and performing antenna setup do take time, but it’s finally started to reach critical mass in several locations and the pressure is on to deliver something that works for them.

I’m looking forward to the challenge.

What’s new in Alpha 8

• Improved peer discovery works off the same method as Babel to be less disruptive to surrounding network environments and more efficient.
• Improved tunnel management now handles every case where a connection may fail to open or be maintained.
• Exit signup improvements, users have to manually sign up for an exit now, but we can reasonably scale without losing Netflix access.
• Lots of usability improvements in the dashboard, including connectivity indicators and loading bars on the right parts of the signup process.
• x86_64 as an OpenWRT platform, make a big router out of an old desktop!

I’m especially excited about this release because it’s finally resolving a lot of the issues that have been keeping us from expanding more aggressively in deployments.

Generally the old peer discovery code was pretty high-touch and sometimes required a couple of reboots before it really behaved. In Alpha 8 that’s all but eliminated, as is the tendency to anger older networking equipment with an aggressive peer discovery strategy.

We’re also on stable releases of Rust and OpenWRT for the first time, as the stars aligned to put all the features and device support we need into the stable branches of each project at about the same time.

In deployments news several of our networks, including Seattle and in Columbia South America are getting their first equipment up soon. Which has in turn driven our push for stable networking in this release.

We’ve also been testing out some cool new equipment particularly MikroTik’s ‘wireless wire’, a line of 60ghz radios with integrated directional antennas that come preconfigured as an L2 bridge out of the box. They are surprisingly easy to setup and perform quite well.

While these aren’t as flexible as many other products the 60ghz frequency essentially makes them immune to the interference issues you get on the normal wifi range and provides very high speeds (full gigabit) over their 200m range. For a variety of situations I could see these becoming our go to recommendation.

I’m hoping that this is the second to last Alpha release, we’ve got one or two more hurdles before payments stuff is ready to be mainlined. After that we’re technically in Beta as we would be unstable but feature complete.

On the subject of development we’re merging some of our longer term feature branches while bringing these new developers on board. So Alpha 7 will be a bit late.

Speaking of releases here’s a rough time-line for features.

Things coming in Alpha 7

• Exit signup refactor: It turns out there was a lot of work between a mediocre sign up solution and something we could expose to the world without concern. The new exit signup procedure is actually sufficient to verify users are human and is generally more robust to failure.
• Live toggling for wan ports: One of our favorite demos is to hook up a few routers and start moving connections around, quickly demonstrating the indestructibility of the mesh routing. But there was one cable you couldn't move, the gateway WAN ports used to bridge traffic out of the mesh and onto the internet were perilously finicky. If not setup perfectly on router start any attempt to use them would fail. Now you can add/remove or change the WAN connection at any time without issues.
• Off-chain payments code: Our first valid eth transaction back and forth is on track for Alpha 8, we hope to be testnet ready by Alpha 9 (that’s early September for those paying attention)
• On-chain payments code: The contract we want to use for our payment channel is already complete but integration and the resulting testing will probably take a while. I’m estimating Alpha 10 before ‘on testnet’ usage doesn’t involve devs bailing out routers burning test coins all the time.
• Peer discovery refactor: Rust is a very great language when it’s safety properties are used well, I’ve been going back over a lot of the original peering code with a fine tooth comb and the goal of higher day to day stability. I’m hoping we’ll see this in Alpha 8.
• Simplified API endpoints: Early on with the router dashboard we used a /settings API to do just about everything, this was great in that it was super powerful and very quick to make, but long term we would like to avoid posting highly specific structs from the frontend. It’s a recipe for tight coupling and spaghetti code. We’ve added simplified endpoints for exit signup already and we hope to have everything else soon.
• Improved and far more helpful dashboard: Right now the dashboard is quite bare, we brought on another front end developer to really polish it from something with the right buttons to something legitimately helpful in setting up and debugging networks.
• OpenWRT version freeze: We keep getting caught by surprise by OpenWRT master breakage, building off of master was fine earlier on in our development cycle but it’s past time to freeze on a commit hash that we test before we update. This way we can just not update when a major device decides the latest OpenWRT build means it should brick itself .

What does ‘stability’ mean for Althea?

I’ve been talking about stability a lot, it might give the impression that our Alpha releases are totally broken. That’s only partially true, while features are often incomplete my personal router is untouched since Alpha 5. It has completed multiple update cycles without the slightest bit of attention.

If Althea was meant to be used in low radio noise environments with a direct fiber line for backhaul like my apartment we would probably consider stability a done deal.

Maybe half of the deployments we’re working with are in technologically modern areas with accessible backhaul fiber, where internet access for consumers is simply too expensive.

The other half live in some of the more challenging networking environments in the world. For these people it’s often not that internet access is overpriced, but that it’s simply not sold or is of such poor quality as to be unusable.

Ultra-high latency, reordered packets, flaky connections, and protocol non compliant devices litter these sorts of environments.

Somewhere in between California and Australia there’s an ISP that will buffer Wireguard packets and play them back out of order 10 seconds later. All while prioritizing ICMP, HTTP, and HTTPS, making the issue almost impossible to detect.

From the perspective of a deployment in Australia this was Althea just not working.

Our first outage report for the Clatskine test network was caused by supposedly ipv6 complaint point to point links suddenly generating spikes of 300ms latency versus their usual 10ms when exposed to pings to ff02::1.

From the perspective of the Clastkine network this was Althea just not working.

Slack and many other websites firewall off TCP MSS autonegotiation. This means they hardcode the segment size of their traffic to a standard 1500 byte frame. It’s extra insidious because protocol complaint websites will work flawlessly and you’ll sit around wondering what’s going on.

From the perspective of any user of that build this was Althea just not working.

I think I’ve made my point. The actual ‘mesh networking’ is just the beginning, if we really want to make a tool that can withstand the sort of environments where people need Althea most it’s not enough for it to work, it has to be indestructible.

We’re attending and presenting at SOON in Toronto this weekend, so keep an eye on the IPFS powered livestream

What’s new in Alpha 6

• DNS servers have been moved to more stable defaults after a recent incident
• Meshing over device WiFi is now a runtime toggle instead of a firmware compile time option
• Dashboard API endpoints are now automatically tested and should generally be more reliable
• Moved to the system allocator for rust Stable builds

Now Hiring!

We have raised a funding round and are now hiring remote developers. See the application here. We’re a fully remote team and we will happily hire wherever the talent is. We are also pretty flexible about pay, so generally expect a little above the average for your region.

In which detailed data confirms the obvious

If it becomes clear that the Zyxel is a part of a larger pattern of substandard performance rather than the exception I’ll start digging into kernel performance profiling and really get to the bottom of the issue.

Good news is that GL-B1300 has a good enough price/performance to become our new go-to midrange pick. Bad news is that’s 100mbps throughput, at least 3x less than what I had estimated the device to be capable of.

True to my word, I dug into the performance at the kernel level on both the WD MyNet n600 and the GL-B1300, trying to get to the bottom of our performance woes.

First off thanks to Brendan Gregg for the wonderful pearl script that converts data from the Linux perf tool into easily readable Flame Graphs. For those of you not familiar this graph represents a call stack, going up the deeper ‘down’ the callstack you go. Length is a relative measure of how much cpu time was spent in a given function.

The below graphs where captured over 100 second intervals during an iperf UDP test from kworker threads using a kernel compiled with as identical as possible flags.

The n600 is a pretty old device, it runs a 560mhz AR9344 MIPS core right out of 2011. Variants of this core are popular into even modern routers. Tested speed with Althea is ~25mbps.

The GL-B1300 is a brand new quad core 717mhz ARM chip Atheros IPQ4028. Tested speed with Althea is ~100mbps.

Considering the process and architecture improvements between these two processors we should be seeing a lot more than a 4x improvement. The latter processor is easily a dozen times more powerful.

The reason we’re focusing on processing power is that in our design with Althea we actually nest two WireGuard tunnels. One to provide security for your traffic as it traverses out to the internet, the other exists between every hop. A very important part of being able to bill for traffic is being able to identify who is responsible for paying for that traffic.

Since each hop pays the hop adjacent to it in our system WireGuard is actually the most expedient way to authenticate traffic is actually from the peer who will be billed. In theory we could improve efficiency by removing the chacha20 encryption and only using the poly1305 authentication.

This is even something we seriously discussed as it became clear even modern devices where not performing to expectations. But I decided to do some more investigating before going down that route.

In the graph for the n600 CPU time dominated by WireGuard tasks. So much so that the iptables and routing rules remain thin little licks of ‘flame’.

But the picture painted by the B1300 is very different. The previously flame thin iptables stacks now make up nearly the entire graph. Packet decryption is relegated to this little corner on the left and an even smaller one about a fourth of the way over. Where you can barely see a chacha20_neon function.

What we’re seeing is WireGuard’s optimized ARM assembly implementation paying off. It’s not so much that iptables tasks are larger on the B1300 as WireGuard tasks are massively smaller.

The similarly optimized MIPS implementation does impressive work for such an old processor. But exactly as expected the newer device blows it out of the water.

So then why is it only 4x faster?

The answer is packet forwarding hardware acceleration. While the n600 achieves it’s sticker 100mbps running stock OpenWRT most modern routers need special drivers to reach their sticker performance.

Testing the B1300 with stock OpenWRT reveals it gets…. about 100mbps throughput without WireGuard. The CPU is easily powerful enough to perform the WireGuard operations, exactly as I estimated.

To be honest I also suspected packet acceleration issues before I began, but it’s good to have rock solid confirmation.

The conclusions we can draw here are that on modern devices Althea’s encryption/authentication stack is efficient enough to not be a problem. Instead we should focus our performance efforts on FOSS drivers for more common packet accelerators.

Fortunately such drivers already exist and are floating around in the depths of the forums. Waiting to be upstreamed, or if that fails applied as local patches to our router firmware.

While this isn’t exactly a great comfort considering that DNS requests are still made in cleartext it’s marginally better than feeding more data to Google.

Or at least I thought. Recently NordVPN decided to move their public DNS server address if they had merely shut the service down the Althea firmware would have failed over to Mullvad’s DNS and been fine.

But instead the service now returns all queries with redirects to the server change announcement.

We have a patch in that will fix this issue in Alpha 6, users will either have to manually delete the lines containing the ip 162.242.211.137 in /etc/resolv.conf and /etc/config/dhcp on their devices or reflash with Alpha 6 once it is available Thursday.

As a response our new default DNS server selection is Cloudflare, followed by Hurricane Electric and L3. While maybe not as privacy focused we can expect that these services won’t be pulling the same tricks any time soon.

We’re still in the process of playing around with DNSsec by default, so we won’t be rolling it out with this fix, hopefully in the next few releases.

What’s New in Alpha 5

• the exits endpoint, curl or visit 192.168.10.1:4877/exits to see detailed network status for easy debugging
• improvements to the update system to better track what’s going on
• some infrastructure for payments
• other bugfixes and stability improvements

The big feature this sprint was a proper refactor of the exit sign up workflow to actually make sense and properly accommodate rare edgecases.

There are many cases where we want to change x or y property and we need a reliable way for user devices to understand that things have been changed without disconnecting them all from the internet until they perform a manual reset.

Now instead of trying to naively sign up to every available exit at once the device will wait for the users go-ahead and provide a reset button for when things go wrong.

Sadly this didn’t make the cut for Alpha 5 as the ui components are not finished and tested.

What did make the cut for Alpha 5 are several bugfixes and a new endpoint the exits endpoint automatically checks all of the mesh properties we would usually check by hand to debug and provides a very easy way to identify problems. In the future it will be integrated into the interface so that users get a short summary of what may be causing their connection problems.

{
“nickname”: “us_west”,
“exit_settings”: {
“id”: {
“eth_address”: “0x0101010101010101010101010101010101010101”,
“mesh_ip”: “fd96::1337:e2f”,
“wg_public_key”: “1kKSpzdhI4kfqeMqch9I1bXqOUXeKN7EQBecVzW60ys=”
},
“message”: “Althea production US exist”,
“registration_port”: 4875,
“state”: “Registered”
}
“is_selected”: true,
“have_route”: true,
“is_reachable”: true,
“is_tunnel_working”: true,
},

Likewise the groundwork for payments is being set out. We hope to have both the exit signup refactor and the first integration of our payments daemon gauc_rs ready for Alpha 6.

In this iteration guac will complete the payments workflow without actually sending said payments off to a full client to be published on the testnet. It’s been an interesting exercise really working out the payment protocol and making sure everything is solid. No huge inventions here, just chipping away at making a solid implementation.

We’re aiming for early August as our target date for having testnet payments online and functioning.

When it comes to device support I’ve had a very confusing week, I added the Zyxel Armor2 to our supported devices list with high hopes of replacing the Turris Omnia at the 300+ mbps embedded range. Only for it to create a new performance per dollar record in the wrong direction. At $160 for a pitiful 60mbps.

What went wrong? The Omnia runs pretty stock Cortex A9, the Zyxel runs a Krait, also an armv7 core which a higher IPC and more processing bells and whistles.

Digging into the OpenWRT port (my primary suspect) the cpu governor is fine and so is the NEON implementation. WireGuard successfully compiles and runs using vector instructions.

The only place the Omnia wins the spec war is it’s DDR3 vs DDR3L memory in the Zyxel. I don’t really think that’s the key component here though, memory bandwidth should be an order of magnitude higher than networking bandwidth.

My primary suspect at the moment is some processor specific patch that was not unstreamed by Zyxel and somehow not captured in the port.

I’ve decided to stop digging deeper on this device and instead look at the GL-B1300, which also has a pretty good spec sheet on paper and ships with a similar processor.

If it becomes clear that the Zyxel is a part of a larger pattern of substandard performance rather than the exception I’ll start digging into kernel performance profiling and really get to the bottom of the issue.

Our head of outreach Deborah will be in Puerto Rico doing a workshop on Althea at the Meshing With Data hackaton later this week and I will be in Toronto for the State of Our Networks (SOON) conference next month. Hopefully we can get good recordings of both presentations to share with you all.

After fixing other issues in the release it became apparent that radio performance had dramatically dropped. Performance tests on the patched Alpha 3 release showed that the performance drop was not due to the software but due to the underlying radio network that Clatskanie ran on.

Packet dumps showed that radios would receive a ipv6 ping to ff02::1 and respond as the ipv6 specification demands, but in the process they would experience a large latency spike.

Delivering out of order packets as well as adding 300ms of latency to a connection that nominally has 40ms of latency. This caused throughput to drop from ~20mbps to 1–4mbps on the same hardware and physical link.

This issue was resolved by updating the firmware on all of the Nanobeam CPE devices as well as the Rocket M5 sector antenna.

Pings to the ff02::1 address where new in the Alpha3 firmware and used for peer discovery. They could be eliminated by re-factoring the peer discovery protocol, we are investigating if these sorts of issues are common enough to be worth the trouble.

By production I mean on real radios and in other people’s homes as a source of internet access. Despite performing admirably in testing everything, of course, immediately started crashing.

Earlier this week we managed to clean up all the race conditions that might cause odd crashes, reduce spam routes that leaked in form exits, and generally polish off a lot of rough edges. That’s all in Alpha 4

But we still had a problem.

Once the crashes where out of the way it became clear that radio performance had tanked. Your first instinct may be ‘well the new software doesn’t perform well’ that’s what we thought too.

But after a series of performance tests we where able to determine that the performance drop can be attributed directly to the traffic over the radios themselves.

It seems that when you send a ipv6 peer discovery packet on some Ubiquiti firmwares it causes a sudden and dramatic drop in throughput and dramatic latency spikes, even reordering of packets. Which does not a happy TCP session make. Definitely the sort of issue you only find in production.

We’re still working on if the solution to this is re-factoring our neighbor discovery protocol to be more restricted or if just updating the firmware on all radios involved is sufficient.

Speaking of those performance tests, we gathered some very interesting data. It seems our current software is mostly memory bound in terms of networking speed. On embedded devices like routers this means you want a lot of cores, the more or higher frequency the better, to handle all the copying. On more advanced platforms like x86 the clockspeed advantage goes away since there is dedicated hardware for memory copying and it simply becomes a memory frequency war.

As a summary a $15 router gets you 20mbps of encrypted throughput, $60 gets you 45mbps and $160 gets 300mbps. I couldn’t really test the throughput of anything faster than with the confidence I would like. I’m satisfied that we’ve reached a decent price to performance ratio now that we’ve got our general network structure finalized.

Deborah is busy on outreach, working on building an Althea community in Portland over the next couple of months.

They met up this weekend for a flashing party, playing around with our latest release and planning a network.

Coming up in the next couple of weeks is Alpha 5, which will include some new features in addition to more bugfixes. As well as an AMA hosted on Reddit’s mesh networking community, this coming Sunday the 17th.

We’ve been working on the Althea governance paper which outlines a platform for creating “subnet DAOs”, blockchain-based organizations which regulate, assist, and promote local Althea networks. Here’s an overview:

The power of decentralization

Althea makes it easy for groups of people and businesses to form local internet access networks, without the network being owned by a single centralized ISP business with all the questions about ownership and management that that entails. Think about it- if there are 4 conventional ISPs in your area, and you are signed up with only one of them, you are only using a fraction of the available network infrastructure. Why? Because existing technology requires that there be one centralized entity with one network for you to pay (unless you are using a free community network, but the lack of money makes these hard to grow and maintain).

With Althea, neighbors can connect to other neighbors, as well as more commercial entities, or even nodes owned by independent entrepreneurs. There’s no requirement for one company to control an entire network to get paid. Every packet follows the most efficient path through the network, and every node is compensated for its contributions to the network. By decoupling compensation from centralized network control, Althea allows for networks that are more redundant, more efficient, and cheaper than traditional ISPs.

We’ve been testing this model in our Clatskanie, OR test deployment, and it has attracted interest from a wide variety of people who want to start Althea networks around the world.

Where is centralized still better?

Centralized networks do still have advantages over the basic Althea model outlined above in a few key ways.

Network promotion: A centralized ISP can devote resources towards promoting their business and signing up customers. In an Althea network, payments go directly to the nodes carrying the traffic. One pattern that we have seen is where a property owner who has a good location for a node (a high rooftop usually) has little interest in going out and finding users to connect to their node.

Network quality control: A centralized ISP is a group of people who benefit from happy customers, since all the money flows through them. If the only payments are for bandwidth, then the only people in the system who will have incentive to hook customers up and keep them happy are people who can expect to sell them bandwidth. There could be a tragedy of the commons where nobody takes care of a network as a whole.

Network security: If one entity controls every router on a network, as is the case with a conventional centralized ISP, then those routers can easily trust their neighbors, and other routers in the network. In the decentralized Althea model, the possibility exists that a router could lie about how good its connection to other routers or the internet is. If there is no factor limiting how many new routers are added to the network, an attacker could even cause routing to grind to a halt by advertising a million fake “Sybil” nodes.

Our solution: Althea subnets

Althea subnet organizations are groups of people empowered to play a role governing Althea subnets. First, let’s review what a subnet is. A subnet is a group of routers who all operate in a common range of IP addresses, and can all route messages to each other. Basically a local Althea network.

Some of the routers in the subnet may have a link to the broader internet. These are known as “gateways”. As long as there are a few good gateways in your subnet, you will have internet access. For this reason, subnets do not necessarily need to be very big. They need to be large enough to connect end users in an area to nearby gateways, but that’s it.

Also, a node can be part of more than one subnet. Most end users will not be part of more than one subnet, but larger nodes that serve end users in more than one subnet will obviously need to be.

Subnet DAOs

Subnet DAOs are decentralized organizations which are empowered to give out IP addresses in a given subnet, and have the option to collect a continous IP address renewal fee from the nodes using these IP addresses. By default, Althea subnet DAOs run on Aragon’s DAO framework, which provides interfaces and contract code for subnet organizers to vote on which nodes to give IP addresses, to appoint other subnet organizers, to manage cryptocurrency and cryptoassets held by the subnet DAO, and many other features.

This system has been designed drawing on our team’s experiences helping run local meshnets. The intended use goes like this: a local group of volunteers, property owners, and entrepreneurs get together to start an Althea subnet in their area.

This group initializes a subnet DAO and adds the members of their group as organizers. Now they can give out IP addresses to people who want to get on the network. They can charge a small fee to each node as well. The fee is paid in the same cryptocurrency that is used bandwidth payments, probably Eth or DAI. This goes to the subnet DAO, and can be used to pay for promotion, meeting space, or other expenses, as well as rewarding the main organizers of the network in the area.

This is important, as it’s common for property owners with good node locations not to be the type to go out and promote Althea, and for people who are good at community organizing and promotion to not own property with good node sites.

Subnet organizers are responsible making sure that users have a good experience on their subnet. This includes helping users get set up, connecting them to hardware installers, making sure they are in range of other nodes, making sure that nodes in the network are behaving reasonably, and stopping attacks on the network.

If they do not do this well, users will complain, and word might get out that it would be easy, profitable, and good to vote them off the TCR.

Decentralized quality assurance: The global TCR

We expect that a lot of the early promotion of Althea will be done by subnet DAO organizers. However, Althea also has a large following of people who find out about it in the media or online, and will need to join a subnet to use the system. For this reason, we need a way to suggest high quality subnets in an area, and to weed out subnets that are providing a bad experience to users.

The global TCR (or “token curated registry”) lets a decentralized network govern a list of good subnets. The TCR model was pioneered by AdChain and has been adopted by many other projects, such as district0x, FOAM, and Civil.

From the Althea governance paper:

Someone starting a subnet DAO can stake any amount of ALGT and be included on this list. Someone who feels that a subnet DAO should not be on the list can put down a matching deposit to initiate a vote to kick them off the list.

If the subnet DAO being challenged loses the vote (because it is known to be blatantly scammy or provide a very poor experience), then that registry is removed from the list, and its deposit is split between the challenger and the token holders who voted to take it off the list.

If the challenger loses the vote, then the reverse happens. The challenger’s deposit is split between the token holders who voted to keep the subnet DAO on the list, and the subnet DAO itself.

Several things can be done with this curated list of good subnets. First, we can use it to populate an online map, or other method for new users to find a subnet in their area to join. Subnets can claim an area of operation. They will then appear to new users entering an address within this area.

If there are multiple subnets active in an area, they will be ranked by amount staked on the TCR, weighted by the size of their claimed area. Subnets with a larger area will have to stake more to have a higher ranking. This is to prevent very large subnets from being able to easily appear at the top everywhere.

Subnets with more staked have more to lose if they perform poorly. Subnets who claim to operate in an area outside the area that their intermediary nodes can realistically provide service will generate user complaints and may be kicked off of the list if they perform poorly enough.

Nodes with standard Althea firmware will not be able to join subnets which are not on the global TCR. This is to stop a bad subnet which is making users unhappy from growing. The organizers could just start another subnet, but it would have to start from scratch and add new users.

What if I don’t like the global TCR?

Althea’s code is 100% open source, under the Apache 2.0 license. We make guides and documentation on how to work on the code and flash routers (aka install the Althea OS) a priority. If you don’t like how the global TCR is being run, you can simply modify the software and flash it onto routers and give them out to people. You could make it so that these routers could join any subnet DAO, even if it’s not on the TCR, or even fork the global TCR if you think it could be run better.

We expect that if the global TCR curation tokens are fairly and widely distributed among people who want to see Althea and incentivized mesh succeed, then the Althea global TCR will be the best way for new users who want cheap, fast broadband internet access to find a local subnet.

As promised for the Alpha 3 release there is now a working exit registration and selection screen, as well as a working wifi settings page.

This marks our first ‘stable’ release, every two weeks from here on our the latest code will be tested and merged into a new release. In sync with this release existing routers will update from the Althea software package repository.

Next week we’ll be transitioning our existing users to this new stable release channel and keeping them there from here forward.

To make the long story short if you’re adventurous you can flash some devices and start an Althea network now. Hook up your neighbor, or just use the system as a convenient whole home proxy.

We’ve also announced the details of our planned governance system.

I really think we managed to achieve the goals I outlined in my inaugural medium post. The protocol is left slim and tokens don’t get involved anywhere that they don’t bring meaningful improvements. Finally and most importantly we create a simple way for users to opt out while at the same time providing easy and safe defaults without centralization.

In other development news features coming up in Alpha 4 include.

• The neighbors webpage working and showing realtime info about your mesh peers.
• A bugfix for potential wireguard tunnel collisions when more than one gateway operates behind on the same NAT (no private data is revealed, on the other hand the connection does stop working).
• An early implementation of full path route verification.

The big ticket item for alpha3 is enough user setup to actually get things done without the command line. Here’s the development version of the dashboard, you should find it in a nightly firmware in the next couple of days.

If you don’t have any OpenWRT capable devices lying around the Linux installer is in far better shape than it was just a week ago and should be easy to play around with. It should be as easy as plugging in an RPI and running the Ansible script. As always hit me up @ttk2 in our development chat. Even simple problems are important because they let us know what we need to explain better in our documentation.

Our outreach efforts continue to line up locations for Althea deployments. Deborah has been developing groups in several cities as well as a playbook on how to connect various types of homes.

Where internal wiring is unavailable or of low quality a ‘reverse tower’ setup can provide building residents 100mb or greater at a tiny fraction of the cost to rewire the building.

If buildings have unused phone lines or coaxial cable they can be used to provide short distance gigabit speeds using G.Fast hardware, we’ve been working with a vendor for a solution around $100 per unit.

Of course for a lot of older buildings the picture of in-wall-copper isn’t so pretty. After snooping around several apartment buildings wiring cabinets. I found that arrangements like the one below are pretty common. This ~600 unit building had at-minimum 100 units on a single unamplified input cable line. With a tree of unpowered splitters per floor, down the hallway, and finally per unit.

Units in this building are lucky to get 20mbps out of their cable modems. Each splitter attenuates the signal and without a powered amplifier and modems struggle to compensate with slower encoding and heavy use of slow error correcting.

After some study and brainstorming I think we could get the whole building to gigabit speeds for less than $200 a unit. By using power adapters in each unit that sent power back up the lines powered splitters could be placed into each box. Reducing signal degradation and allowing each unit to use the full line capacity.

After that it’s a matter of cutting a few levels off of the tree and dropping in routers per floor or at least per wing to bring the oversale ratio down.

Installations in single family homes are more a matter of land layout than anything else. It’s all about height and line of sight with none of the tricks you can pull in denser living situations. That being said the lack of having to rent roof space really helps.

All of these potential mesh groups are a little different, some have backhaul but lack a solid customer group, others are the opposite. Some are municipal internet advocates trying to make local cooperatives. We’ve been working with our contacts to make sure that each of them are successful.

All this development has led to a few interesting bugs to chase. On the EdgerouterX I’m chasing a bug with UBIFS where UCI config files become corrupt without a explicit call to ‘sync’. I’m on the verge of just putting a hack into the startup script for this, but I’m going to spend a few more hours trying to track it back to the source.

In trying to get the Edgerouter Lite working I tracked down a bug in Rust’s generated assembly code the assumption of hard FPU support seems to be a recurring theme in the cross architecture toolchains they have available.

I also spent about 4 hours hardening the exit installer playbook for what turned out to be a local hardware issue. My Gil.iNet ar750’s wan port started flaking out and producing 70% packet loss at all times, this resulted in me seeing the negotiation packets on each end but not realizing the session was being screwed up by lost packets.

While there are still plenty of strange bugs floating around we’ve reached the point where the nightlies tend to work more often than they don’t. Putting us well on the way to a truly stable release.

Nestled in the dense foliage of fir and alder trees in the Oregon forest, a home built radio tower reaches for the sky. This tower is the novel design of small town computer programmer and tinkerer, Shaun Ripplinger, and was built to test the new incentivized networking software developed by Althea. The Althea system is a novel form of decentralized Internet Service Provider (ISP), with automatic payments and encrypted network traffic. Instead of buying internet from big corporation like Comcast or CenturyLink, communities like Clatskanie, a town with a population of around two thousand, are empowered to start their own neighborhood internet services.

Here in Clatskanie, there is a small grocery store, one stop light, and everyone knows each other. And, like most other small towns, they suffer from poor broadband access and few choices. In fact, according to the FCC, 39 percent of rural Americans (23 million people) lack access to broadband internet. Community Network enthusiast and head of the Cascadian MeshNet group, Deborah Simpier explains, “Running a small business here, l can see firsthand how this lack of access affects my community and its members. People have to arrange their days around access to the internet, driving into town to use the internet provided at my shop or at the library. Parents arrange their home school schedules to start at five or six in the morning when the satellite internet has no data caps. People struggle to go to online school or keep up with online meetings for work.”

It is not only the lack of access that concerns members of this small community. The looming threat of the end of net neutrality brings concerns of censorship and throttling to the minds of small business owners like Joann, who has an online store selling live plants and homemade soaps. Joann wonders if online businesses like hers could compete if ISPs make fast lanes only for business that can afford to pay higher fees.

It is these concerns about the future of the internet and the lack of access that propels the members of this community to look for alternatives beyond the current rigid framework of corporate ISPs. Centralized ISPs have little financial incentive to build costly infrastructure to outlying, sparsely populated areas. With a decentralized model like Althea, however, many nodes add robustness and fluidity to the outlying branches of internet service. This decentralized model not only adds service and choice for those in rural areas, it can add resilience in disaster situations, as members of the network can communicate with each other even if outside internet access is limited.

Clatskanie’s decentralized internet starts with sector antennas mounted at the computer shop in the center of town then broadcasts internet to linked antennas at nearby homes and businesses. A radio tower high on the hill above town rebroadcasts the signal to residents on the other side.

Building new internet infrastructure is not without its challenges, however, and traditionally becoming an ISP required a sizeable investment and specialized skills. Althea’s new design tackles these challenges with automatic cryptocurrency payments; secure, encrypted connections; and built-in VPN services that mitigate much of the liability of reselling a connection. It was these improvements to a complicated service that inspired Deborah Simpier and her neighbors to see the possibilities of a decentralized, community based ISP. “What we build here to support our neighbors and uplift our community, we hope will be a model for other cities and even urban areas to follow, “ Deborah says. “We also realize that hardware like radio towers and antennas are not designed for non-professional use, so we are working to build things that are more accessible for the home or business owner.”

To meet these hardware challenges, members of Cascadian MeshNet have been working on an easy to set up radio tower for line of sight internet they call a Rapidly Deployable Tower System or RDTS. Their first design is a expanding scissor lift, which they have deployed on the top of a wooded hill above town. The radios and router are off-grid, powered by solar, which is a challenge in the rainy climate of the Pacific Northwest. Like many innovative projects, the tower is a work in progress, along with the solar power which suffers many outages in a climate that the Seattleites would find themselves comfortable in.

Members of this rural community are not thwarted by difficulties though. Like many rural people they value individual and community self-reliance; working together to solve problems. They have installed a number of households with plans to expand further into the underserved areas along the Columbia River and in the heavily forested area of Mist Mountain as they test Althea’s new people-powered internet. If you blink, you might miss this one stoplight town, but if you meander down a forested lane or look up at the houses on main street, you might notice a parabolic dish or an antenna — a symbol of this small communities hope of a better internet.

Alpha 2 marks that milestone, from here on out nightly firmware releases will be available and when flashed they will auto configure with our test exit and provide internet to you and any other members of a local mesh.

From here on out it should be pretty straightforward for anyone to flash and play around with Althea. Nightly firmware images and flashing instructions are available on our firmware page.

What’s New

• Exit autoconfiguration
• User credential autogeneration
• Traffic watching and accounting (it will generate a bill/invoice, but won’t pay it)
• Peer discovery and negotiation of secure peer tunnels
• Usable nightly builds: available here
• Ready for full time use by the adventurous
• User interface extensions and improvement
• A real exit signup process
• Load statistics gathering and the requisite opt in/out button.
• Automated exit load balancing client side
• Package updates pushed daily (so you don’t have to reflash the nightly to keep up)
• Everything we think is working but will inevitably end up being actually broken in Alpha 2

The short term plan is to get Alpha 3 out the door and then try and get as many users as possible online so that we can hash out bugs in parallel to the development of the real billing code. We won’t be actually charging people for service yet, but we will be watching carefully to make sure the accounting is accurate, so that we can deploy real payments with confidence when they are ready.

For a lot of people I expect Althea will be attractive simply as a whole home VPN solution that just to happens to make it very easy to hook up your neighbor.

On the more business side of things we’re at EDcon in Toronto this week watching Vitalik dance and meeting up with the local Toronto Mesh group TOmesh.

Our head of outreach Deborah has been organizing local mesh groups in Vancouver BC, Tacoma WA and Portland OR. The biggest challenge is finding good resalable backhaul, it’s a business to business market unused to amateurs. We’re working on making partnerships to simplify that.

Last week we showed off the off grid tower Cascadian Meshnet has been working on. It survived it’s first storm (~50mph gusts) with no damage.

On the software side things are coming together for our next major revision. A big feature for this revision is a pretty barebones user interface. It hits all the major points for configuring basic options and seeing what’s going on with the mesh.

A good deal of the work right now is just getting the router to serve the dashboard webpage and then allowing that dashboard webpage to query all the information it needs to present to the user. Feedback on desired options is welcome.

Our bet on Rust continues to pay off, the current router application has a great runtime footprint, even using fully featured libraries for handling http requests and threads it still only requires 20mb of ram.

If we could only get that binary size below 2mb it would be perfect. We may yet make that happen using a binary packer or Xargo.

In general we’re in the integration and polish phase of this release. I’m hoping to be ready to release the next version of Althea’s firmware in a month or so.

This version will be what I consider to be late Alpha. With all the features required for easy setup and basic use. But no functioning payments.

The last thing we’ve been working on is supplier relationships. The logistics challenge of buying, flashing, and then shipping out routers is time consuming and somewhat expensive. Our goal is to have a smooth transition to suppliers, who will be better able to handle the logistics challenges of moving hundreds of devices so that we can scale quickly.

Lots of moving pieces involved in building the infrastructure of the future. But I think we’re staying on top of it ok.

We’ve had some downtime in Clatskanie trying to use a cheap n600 as the gateway infrastructure, I was curious how it would react under prolonged load. The answer is not well, it’s being swapped with an Edgerouter.

Sadly it doesn’t look like the n600’s are going to have a very long shelf life. After extensive debugging I have found their performance issues are due to slow context switching on their single core cpu’s. They simply can’t finish decrypting WireGuard packets fast enough.

We’ve also found some really bizarre WireGuard issues with our Australia test deployment. The backhaul tunnel to the exit server goes on and off every couple of seconds. I think it’s related to packet reordering on the backhaul network if I had to guess. More detailed investigation is in work.

The Claskine deployment will be working on getting their off grid tower deployed and linked into the mesh on Wednesday, I hope to get some cool pictures from that to share with you.

Way back when we talked about how promising on chain governance was.

You may have seen the drafts floating around the Matrix channels, but it’s essentially a pluggable governance architecture where people can create an administrative system for their local mesh group that raises a fund pool for common works and deals with administrating the local mesh. Exactly what the voting rules and such are is up to the users.

There’s also a global level designed to maintain quality control and direct some funds to the company. Essentially a TCR where you pay to register your deployment for use by our off the shelf devices. It’s a good way to decentralize some of the more closed parts of having a real Althea product on shelves.

It’s a legitimately difficult task to both plan to generate revenue and ensure that the product is still fair and ‘free’ as in freedom to user. It’s a hard balance and I’m sure some people will be upset no matter how we do it. But I’m happy with the one we’re striking.

You’ll see a nice summary and a official spec to comment on once we stop editing half of it every week. Or you can find the in-progress version in our Riot dev channel.

Two are using CPE’s as pictured below for long distance links, of a few yards to a mile. The other two are connected using the 5ghz wifi in the routers themselves, which is performing flawlessly over short distances with only 3 peers.

These are representative of our two major use cases, high density housing where short distance ‘pure mesh’ is possible and long distance communication using specialized hardware. In an apartment complex for example a home router would be able to resell to adjacent units above and below. Then specialized hardware would be used to reach beyond that.

The key leverage of mesh technology in Althea is not to create a network without infrastructure, but to create one out of any infrastructure available automatically.

Keeping up with the influx of new interest is really cutting into development time for me. It’s fortunate we already have one employee up to speed, but as the resident cross compiling and tooling expert our on router tooling and specifically process improvements for new deployments are a high priority for me.

The faster I can start posting ready-to-use firmware files on our website the quicker users can move to flashing their own routers without my intervention. Which is currently required not because there isn’t automation, but because that automation is too environmentally delicate for a non-expert to setup.

The router software now has auto setup features to fix this. So it’s a matter of getting it properly integrated into the firmware and our production tooling. We also have the first iteration of our billing software code complete, so it’s undergoing integration testing with the auto setup software.

One of the big challenges is updating core network software on remote devices in such a way that they will come back online and remain compatible with the rest of the mesh. Since a split-version mesh without interoperability will isolate devices that need the mesh to download the update.

These catch-22’s require manual human intervention, so we’re going to take even this first update cycle pretty carefully.

It feels good to be making steady progress. Especially since we keep seeing our engineering decisions pay off.

The goal is to help bring together people who may be interested in using Althea in an area as well as show our expansion as we get some of these locations actually online.

There’s a margin of error to prevent revealing home address of course, we only take city and zip.

It’s probably a little premature considering snow and the need for a taller pole has delayed the first few users in Clatskanie and I shipped routers to the other two deployments just today. But I’ll consider deployments with gear on roofs to be ‘active’ for now.

In our last update we shipped our first devices, I had been growing concerned with our dependence on these WD n600’s and n750’s especially as we began to realize some of their performance limitations.

Using Babel’s excellent in kernel forwarding design even the bargain bin n600 can hit line rate and mesh at 100mbps on several ports. But the encryption overhead for the Wireguard tunnel is non trivial and peters out at about 30mbps. Exactly why is a bit of a performance mystery, there’s about 50% free CPU time but no tuning I can find seems to improve it. Not bad for $16, but there’s untapped potential there.

What this means right now for Clatskine users is that they can resell to two other homes before they even cut into their own bandwidth.

In the search for better hardware the firmware builder now has support for the GL-AR750, Edgerouter X, and UniFi Lite. The AR750 was disappointing, for $25 it mirrors the performance of the n600’s since they share a chipset. The Edgerouter on the other hand makes a great Althea device, with gigabit speeds on the mesh and about 600mbps over Wireguard. The Lite is somewhat harder to test conclusively with the involvement of WiFi but breaches 200mbps over WireGuard easily enough for me to be unconcerned.

While the Edgerouter is great for network backbone tasks it’s lack of WiFI eliminate it from the running as an ideal home device. The UniFi lite has the opposite issue, great wifi, but with only one Ethernet port it won’t satisfy power users or have a ‘plug and play’ resale capability.

The quest for a under $100 device with no performance compromises continues.

On the software side the billing daemon is coming along nicely. I’ve been working on integrating some more bootstrapping support into it as we try and bridge the gap between the development and testing environment and our real operating platform on the routers.

WireGuard is once again a major focus of concentration here, by design WireGuard requires out of band key exchange and ip address assignment, in exchange for this you get great speed, simplicity, and other security properties but it’s something of a pain right now to setup a new device.

Currently a device is flashed with a blank firmware and is then customized by running a shell scrip at first boot. This shell script then drops the required credentials to be sent to the WireGuard server (currently via copy and paste), while this works it complicates the flashing process.

Making the flashing process simpler is a long term project with the eventual goal of a push button ‘install Althea’ application. While maybe still not end user accessible it would be great if the people setting up Althea on the ground could easily flash devices from the box instead of us having to ship them around.

Not only did I get to see the Falcon Heavy finally fly from the cape, but I also finished the Althea alpha firmware.

Right now OpenWRT devices flashed with Althea firmware using our firmware builder automatically connect to the local Babel mesh network. They then create a secure WireGuard tunnel to exit.altheamesh.com which all internet traffic is put through.

There’s no extra configuration required beyond connecting the devices to another mesh node by whatever method is available, just plug-in and instant secure internet access.

Our first devices are WD myNet N600’s a now defunct but reliable and cheaply available fast Ethernet (100 mbps) router. My own tests show that a user can expect to get 30/50 mbps over the mesh, I’m hoping to ratchet it up to line rate with some additional optimizations.

These devices are packed for transit to Clatskanie Oregon where members of Cascadian Mesh are wrapping up antenna installation. In this first phase of the deployment 6 users are being installed with taller masts and higher powered antennas. Using this extended line of sight these nodes will form the backbone of a network by reselling internet to their neighbors.

Chances are we’ll have to swap out these n600’s for faster devices within the year. If the network grows quickly these backbone units will get overloaded. But that’s a good problem to have, especially when you’re being paid to pass traffic.

Speaking of payment, Rita our billing daemon is taking shape and should be ready to push to these devices by our next update. That will mark our movement from testing basic networking structure into testing billing components.

Our short term goals focus on two things

• Getting billing and payments finished, tested, and audited
• Making the deployment experience as easy as possible

Likewise Rita our billing daemon continues to evolve rapidly to meet our Alpha milestones.

My immediate goal this week is to get the MVP firmware routers physically shipped for initial testing, this should coincide with the first major links being tested in Clatskanie OR. I’m quite happy with the firmware at this point, most of what remains is infrastructure setup like update servers and the networking exits.

There’s also a level of burn in validation, 24 hour packet loss tests etc are important to prove out the reliability people expect from their internet. I’ve found a few issues so far that only show up after multiple hours of uptime. I’m glad to get those solved now and not in the field.

One of the more interesting things CascadianMesh is working on are off-grid easy to deploy towers. Setting up towers can be one of the more expensive parts of running a WISP or Mesh network, unless you can find and pay for space on an existing building or tower instead.

In the Clatskanie network map there’s a ridge separating several users. They plan to bridge it with an off grid solar powered rapidly deployable tower of their own design. You’ll be able to read more about it soon on their blog.

On a more personal note, my last day at Red Hat was last Friday, so I’m now full time on Althea. We will also be bringing on two other software developers in the next month or two bringing us to a total of 4 full time developers to move from Alpha to Beta stages before mid this year.

There’s a lot to do before we reach Beta status and I’m looking forward to being able to give it my fully attention.

After overcoming several different compilation and size issues we’ve mostly achieved our goal. While Rust binaries are still very big by normal embedded device standards (on the order of 2mb) they are small enough that we can ship full featured applications developed without restrictions and have them run well.

This guide is about issues with advanced syntax and building more complex crates, if you want to cross compile a simple ‘hello world’ program start with the excellent rust-cross guide.

proc_macro

High level abstractions with low level results are littered through out the Rust language. In this example we’ve defined serialization for the base types and then can simply automatically derive it for all objects containing those types.

#[derive(Debug, Serialize, Deserialize, PartialEq)]
   struct MyStruct {
       addr: EthAddress,
       sig: EthSignature,
       key: EthPrivateKey
   }

This compiles perfectly on normal x86 targets, works perfectly too. Everything seems perfect until:

[justin@aperturescience.robot serde]$ cargo build --target mips-unknown-linux-musl   
Compiling proc-macro2 v0.2.1   
Compiling serde_test v1.0.27 (file:///home/justin/repos/serde/serde_test)   Compiling quote v0.4.2
error[E0463]: can't find crate for `proc_macro`  --> /home/justin/.cargo/registry/src/github.com-1ecc6299db9ec823/proc-macro2-0.2.1/src/lib.rs:27:1   
|27 | extern crate proc_macro;   
| ^^^^^^^^^^^^^^^^^^^^^^^^ can't find crate
error: aborting due to previous error
error: Could not compile `proc-macro2`.
warning: build failed, waiting for other jobs to finish...
error: build failed

What is proc_macro and why doesn’t it exist when compiling for targets that aren’t x86? proc_macro is a compiler plugin system. Since Rust allows crates like Serde to provide syntactic sugar that gets turned into actual code the compiler structure has to accommodate code generation from a different source than itself.

When you go to compile a Rust program with #[derive(Serialize)] or some other property provided by a crate like Serde you’re building a proc_macro crate, these special crates generate a compiler plugin that’s used at compile time to build the code that #[derive(Serialize)] translates to.

So trying to crosscompile a proc_macro crate doesn’t really make any sense. You only need the plugin to generate code and all of it’s functionality is useless on the cross architecture you’re building for. Therefore Cargo just tells you that this useless version of proc_macro doesn’t exist.

This is caused by having a specific type of useless dependency. If you’ve tried to extend or define other proc macro traits you may have a extern crate quote and or extern crate syn somewhere that’s no longer needed.

This can happen if you wrote some derivations that used proc-macro functionality and then re-factored them out without removing the dependency import. Since you aren’t defining anything that actually needs a macro the compiler will let you leave off the required tag in the Cargo.toml.

Inside of this edge case you will get the above error, it can be corrected by either removing the creates quote and syn if they are no longer needed or properly defining and using a macro in the crate that imports them.

Big thanks to Manish Goregaokar for helping improve this section of the article.

Linking

With the proc_macro issue gone we can try again. Only to see this:

error: linking with `/home/justin/repos/althea-firmware/build/staging_dir/toolchain-mips_24kc_gcc-5.5.0_musl/bin/mips-openwrt-linux-gcc` 
failed: exit code: 1 |  = note: "/home/justin/repos/althea-firmware/build/staging_dir/toolchain-mips_24kc_gcc-5.5.0_musl/bin/mips-openwrt-linux-gcc" "-Wl,--as-needed" "-Wl,-z,noexecstack" "-L"

Followed by a horrifyingly large number of libs. But the key line is at the bottom.

/home/justin/repos/althea_rs/target/mips-unknown-linux-musl/debug/deps/libminiz_sys-9df090e5aeaaf6af.rlib: error adding symbols: File in wrong format

Oh no, an x86 library has found it’s way into our mips linking operation! Cargo was building and linking for the right platform but somehow those flags got ignored for some subset of the libraries.

Crates like cc-rs are used to link external C or C++ libraries into Rust programs at compile time. You might notice that the popular rust-openssl crate uses this method to link against the system OpenSSL.

It’s not enough to tell Cargo what to compile for, you must also let cc-rs know that it’s cross compiling and rust-openssl needs to know where the target OpenSSL headers are located.

#!/bin/bashexport CARGO_TARGET_MIPS_UNKNOWN_LINUX_MUSL_LINKER=/home/justin/repos/althea-firmware/build/staging_dir/toolchain-mips_24kc_gcc-5.5.0_musl/bin/mips-openwrt-linux-gccexport TARGET_CC=/home/justin/repos/althea-firmware/build/staging_dir/toolchain-mips_24kc_gcc-5.5.0_musl/bin/mips-openwrt-linux-gccexport HOST_CC=gccexport MIPS_UNKNOWN_LINUX_MUSL_OPENSSL_DIR=/home/justin/repos/althea-firmware/build/staging_dir/target-mips_24kc_musl/usr/export PKG_CONFIG_ALLOW_CROSS=1cargo build --target mips-unknown-linux-musl --release

Notice how in this build script we not only specify CARGO_TARGET but also TARGET_CC and HOST_CC which are read by cc-rs to determine what to compile for where. Finally we specify the OPENSSL_DIR for our target architecture. This script can be simplified some, but is useful to illustrate all the moving parts clearly.

Building natively

Now that we have our binary compiled how do we get it onto a OpenWRT device? The easiest and best integrated way is to build as part of a build system feed using a makefile.

You can find ours in the althea-packages repo with the final and most minimal set of flags you need to compile.

define Build/Compile	
(\		
cd $(PKG_BUILD_DIR) && \\		
PKG_CONFIG_ALLOW_CROSS=1 \		
OPENSSL_DIR=$(STAGING_DIR)/usr/ \		
RUSTFLAGS="-C linker=$(TARGET_CC)" \		
TARGET=$(RUST_TRIPLE) \		
CC=$(TARGET_CC) \		
CFLAGS="$(TARGET_CFLAGS)" \		
TARGET_CFLAGS="$(TARGET_CFLAGS)" \		
CXX=$(TARGET_CXX) \		
CXXFLAGS="$(TARGET_CXXFLAGS)" \		
TARGET_CXXFLAGS="$(TARGET_CXXFLAGS)" \\		

cargo build --all --release --target $(RUST_TRIPLE) \	
)	$(STRIP) $(RUST_BIN_PATHS)
endef

Be sure to include the binary stripping step and use some of the less invasive options to shrink binary size in this guide.

We can’t guarantee that these steps will work for all Crates, for example our makefile may need HOST_CC defined if cc-rs required it. But we wanted to document how we managed to get no compromises Rust code onto OpenWRT. Fulfilling the promise of a high level development environment with low level outcomes.

Althea has a 2-layer model. From the whitepaper:

The base primitive that Althea is built on is a pay-for-forward network. If all the mechanisms in the preceding sections work correctly, we have a network where nodes can pay each other very granularly for the service of forwarding data, and verify that the forwarding is happening correctly. This section deals with using such a network to provide the one of the most popular network services, internet access.

Basically, Althea functions a little like the postal service. In the postal system, you pay some money to send a package somewhere. In Althea, you pay some money to send a data packet to a destination on the network.

In the second layer of Althea, nodes provide services on this network. The most popular service, and the only one we are currently working on, is the service of internet access. This is provided by Althea exit servers.

A user selects which exit server they would like to use, and then opens an encrypted tunnel with them. The user node pays for packets containing internet requests to be sent to the exit server. The exit server forwards the request on to the public internet using network address translation (NAT).

When the exit server receives the response, it pays to send it back to the user node originating the request. But where does the exit server get the money to pay? The user node has opened a payment channel with the exit server. This allows it to pay for the data that the exit server is sending back to it as a result of its request.

Accessing other networks

It’s easy to see how this architecture could be extended to support types of exit servers that provide access to other networks. Projects like IPFS, Blockstack, Filecoin and Maidsafe are all working on “overlay networks” which provide new ways to access data that is usually stored on centralized servers run by big corporations. All of these networks can be run over the existing internet, but in the future they aspire to be integrated at a deeper layer.

By integrating into Althea at the exit server interface, they can provide a more efficient way to access data held in these networks. Normally, a request for data from IPFS might have to go through an exit server, over the internet, to an IPFS node. By building the IPFS node directly into the exit server, this level of indirection can be avoided. It might have direct leased line fiber connections to other IPFS nodes, or other more efficient arrangements. I’m not an expert on IPFS.

Additionally, IPFS, and most of the other systems listed above have an element of caching. This caching means that if a certain piece of data is requested twice from the same IPFS node, it may be able to serve it directly out of memory or storage the second time. This will increase efficiency, especially in areas which already face challenges getting internet access.

Alpha Deployment Status

In our last update we hoped to have the minimum viable firmware done to roll out to our Alpha users. Not quite there yet I’m afraid, we spent a lot of the last two weeks trying to raise a seed round and it was exhausting beyond words, although coming to an end now. Our partners in Clatskanie are similarly delayed with network hardware setup by a cavalcade of personal mishaps and bad weather. Overall progress continues and we don’t expect to be too far delayed, we may still hit the end of the month.

proc_macro woes

Althea selected Rust as it’s major implementation language several months ago after a lot of discussion. Other modern languages such as GO, Python, or Javascript would all consume too much memory and disk space pretty rapidly. Especially if we developed by pulling in lots of libraries.

C and C++ would be both more performant and small enough but lacked modern safety features. There are developers who can write great cryptography and blockchain applications without these safety properties through heroic effort and care. But we would prefer to avoid the risk.

In comes Rust, a modern compiled language that promises the best of both worlds. The safety of a modern language but the speed and size of C/C++ even when using large libraries.

Well obviously I’m talking about problems, so what went wrong?

proc_macro is a very cool part of the Rust language, it allows compile time syntactic sugar to be turned into static and efficient compiled code.

#[derive(Serialize, Deserialize, Debug)]
struct Point {    
  x: i32,    
  y: i32,
}

It’s great when it works, which is all the time on desktop. Sadly for OpenWRT targets we’re getting that new language smell.

Unable to find crate `proc_macro` on musl target · Issue #40174 · rust-lang/rust
Using procedural derive macros on musl seems to work just fine for the most of the cases, but there is one pitfall; in…github.com

Use of proc_macro is very pervasive because it’s just so darn convenient and great. Working low level magic with almost no developer effort. After evaluating our options we’ve decided to make a serious attempt at fixing this in Rust upstream.

If that fails we can always just avoid using code that triggers this issue. But it would be really great to live the dream of no compromises embedded development. Even if we have to do some work to get there.

Big thanks to contributor drozdziak1 for doing a lot of the investigative work here.

“Why not do a token sale?”

I’ve been asked this question countless times since beginning work on Althea, as have my co-founders. This question became more emphasized, however, when we began our funding stage. As the first question interested investors asked, we realized Althea needed to clarify why our answer to that question is no, we will not be holding a global token sale

Vitalik wrote an excellent blog post about the ideal valuation for an exchange token back in October. I strongly suggest you read it, but the point is this: the less general purpose a token is, the shorter its usefulness for users; but for speculators, the longer they hold the token, the higher the price.

This conflict in goals pushes users to competing protocols or protocol forks as outlined in Teemu Paivinen’s blog post on thin protocols.

For Althea, our focus is providing cheap, fast, free (as in freedom from internet monopolies) internet service to communities globally. As a result, we tried looking at a token sale through the lens of an Althea user, and to the end user, a token sale would need to provide them with something significant that couldn’t have been achieved otherwise.

The internet is such a universal platform that any attempt to capture significant value would be immediately forked and replaced unless that value was in some way returned to the user.

What is the value?

The value to the customer is internet service, delivered well and paid for directly.

We spent a good deal of time considering the ‘mine for bandwidth’ problem. In principle it’s perfect, it provides a speculative incentive to contribute and encourages even small nodes to participate.

Unfortunately, it turns out to be inordinately complicated. To prove that useful bandwidth was actually provided on chain, information about the entire mesh needed to make several round trips to the blockchain to complete a single round of payments. Building this technology out is currently not feasible and will not be for many years to come.

So going back to the end-users, we then realized that they didn’t care if they could ‘mine for bandwidth’. If they can be paid for bandwidth contributions directly, to them, it makes no difference. This meant we could simplify many parts of the system if cryptocurrency became a simple payment. This also means that interest in mining bandwidth wasn’t to provide better service to end users but rather because of the prevailing ‘token first’ thinking.

Incentive hell

We are big fans of Open Source, because everyone, including the developer and end user win. The Open Source community is one where existing contributors are encouraged to mentor new members in exchange for receiving new contributors and new ways of thinking.

The technology sector has avoided stagnation because of technologies that have come out of the Open Source community. It gives individuals everywhere the ability to compete with industry giants on equal footing enabling them in some cases to disrupt monopolies that had been in existence for decades.

Sadly, token projects don’t enable this type of joint community participation. When huge value is delivered by early speculation, developers are not incentivized to develop the code. Instead, they may find their time spent pumping the token or building new token features to wow speculators.

Issuing a token or asset for a project as it pertains to the project’s functionality, isn’t a flawed idea. The problem comes when huge incentives are given to shoehorn a token sale into projects that then compromise or complicate the core function of the project and impede users.

There are a few problems in Althea that may be good targets for a token curated registry or some sort of on chain governance model for local mesh groups. But a rushed global ICO would be counterproductive such future measures, putting on chain work before we even knew what a working deployment looks like.

When everything is a race to issue a token the only winning move is not to play.

What does this mean for Althea?

We decided to make Althea a cryptocurrency agnostic platform. Starting out, we will use Ethereum and ERC20 for token support. But as we grow, we will add payment plugins for other cryptocurrency platforms in order of demand. As an open project anyone is free to contribute support for any cryptocurrency into mainline Althea.

We’ve spent time thinking about what it would take to really decentralize internet service, provide real value communities, and how to align our business model with that.

This is our plan:

1. Create a strong open source community to decentralize development
2. Use commodity hardware to reduce costs and increase user freedom
3. Run a lean, mean series of auxiliary services
4. Phone service and in home support for end customers
5. Sell pre-flashed re-branded consumer devices
6. Run a trusted exit node (takes a small fee) for these devices
7. Provide various information to mesh entrepreneurs about network state and status.

What they are not allowed to do is to prioritize traffic going to or from different destinations on the internet. This is to stop them from making Hulu really fast, and Netflix really slow, thus driving Netflix out of business. This would be useful to the ISPs if they owned Hulu, and harmful to consumers if Netflix was actually a much better site. Both of these things are arguably true.

The root issue here is that in many areas, the big ISPs are the only option. This is what gives them the potential power to unfairly choose winners as in the example above. Althea is designed to bring increased competition into the broadband market by enabling communities to set up their own mesh networks.

Althea’s payment routing protocols enable each piece of equipment in the network to collect its own payments for providing service. You can think of this as “mining bandwidth”. Since each participant is paid by the protocol to keep their networking equipment in good shape, you don’t need a top-down monopoly to collect the money from subscribers and control the whole network. This is similar to how you don’t need a top-down banking system for Bitcoin to work.

This payment technology will make mesh networks competitive with the existing ISP monopolies. People will be able to switch away from the monopoly ISPs if net neutrality loses out and they start throttling Netflix in favor of lame-ass Hulu.

But I hear you asking- “can’t a node in an Althea network go against net neutrality”? Because of Althea’s architecure, it’s actually very difficult for nodes not to be neutral. You can read more in our how it works section, but basically, a core part of Althea’s network is the “exit node”. The exit node is chosen by the user and is a lot like their ISP. The key difference is that in Althea, the exit nodes are not owned by the same people who own the physical network. The physical network is owned by the neighbors in a community, while the exit node is run by an exit node company.

A user’s internet use goes through an encrypted tunnel run by the exit node. This means that the nodes in the physical network cannot spy on the traffic, and they also cannot prioritize Hulu over Netflix. The exit could be non-neutral, but since they do not own the physical network, it’s very easy for the consumer to switch to a different exit node. The decentralized ownership of a Althea networks is a big part of what will keep them neutral and fair.

A few weeks ago Rightmesh, a cell phone mesh product, published their technical paper which provides an interesting alternative architecture with very different focus from our own.

In this post we’ll be comparing our products and contrasting different vulnerabilities, performance, and implementation challenges that come inherent to each product. We will also identify several potential problems in both Althea and the version of RightMesh described in their paper and suggest how they could be addressed.

Principles

This commentary is somewhat limited by the fact that RightMesh publishes has yet to publish any cryptocurrency-related code on their  Github. Unlike Althea, large components of the RightMesh software stack will be closed source. They will publish the cryptocurrency-related components while using a proprietary mesh networking protocol from their parent company Left.

We publish our code as we write it. The Althea Github contains all the code we’ve written and everything you need to build the systems we are using from scratch. We are committed to open source.

Regardless of our principles, without open source specifications we have to infer the capabilities of Left’s mesh protocol from off-hand claims made by RightMesh about its capabilities. We’ll try and be clear about the assumptions behind our descriptions below, pending feedback from RightMesh.

Target devices

Althea’s primary goal is to lower the cost of bandwidth for everyone. To that end we’re very focused on home internet right now. While the mobile space is compelling, we’ve chosen to focus on bringing competition to the more stagnant home internet market first. Our software is designed to be run on Linux-based routers connected to stationary high-bandwidth wifi transmitters, or runs of cable and fiber.

On the other side of this decision is RightMesh. RightMesh has chosen to go all-in on smartphone software. Smartphones are not designed to route large amounts of traffic, so this focus naturally shapes the usage of their system. RightMesh is more about allowing dense swarms of cell phones to extend the range of a wifi hotspot or cell tower, while Althea is about changing the economic model of home and office internet service. A swarm of cell phones will be able to extend the range of a cell tower or wifi hotspot by a few hundred feet, while the high-bandwidth networking hardware Althea is designed for gets the internet connection from a data center to an entire town or city.

It’s this hardware decision that drives the design differences between RightMesh and Althea. A few examples: Phones do not offer much access to the underlying system, and most of the networking code must be rewritten as comparatively inefficient “userspace” code. Routers typically can be programmed from scratch and offer access to the lower levels of the very high performance networking code in the Linux kernel. Phones have omnidirectional antennas which cannot send signals as far in any particular direction, and in dense areas can interfere with each other. Much of RightMesh’s design deals with this fact, and they are forced to reimplement a lot of core networking code. Althea is designed to run on the kind of focused directional wifi beams, wires and fiber that provide internet service today, and will always be the best option for high bandwidth and low latency. Because of this, we’ve been able to leverage an enormous amount of existing, proven, and very fast networking code.

However, taking a phone-first approach is exciting, because it could bring access to very remote areas or disaster zones where nobody has had a chance or ability to install networking equipment.

Routing security

RightMesh uses Left’s proprietary mesh protocol. However, there is not a lot of information about how it works. We do know that it is a hop-count based protocol:

Currently, RightMesh selects routes based on the least hop count path. At launch, it would be updated such that there is equal weight between the fewest hops and lowest cost.

They intend to one day add other metrics, such as latency, packet loss, etc. However, no details are given about how this will be accomplished.

In time, this algorithm would become increasingly complex so that performance of the mesh may be fine tuned for individual users where a best path uses a combination of speed, cost, performance, and even reputation of participating nodes.

To securely use a metric other than hop count, RightMesh will have to add some kind of verification to their routing protocol. We’ve found this to be one of the more important challenges of incentivized mesh networking, but RightMesh has not shown any techniques to do it. We’ll explain in more depth after a bit about Althea’s routing.

Althea uses a modified version of the Babel routing protocol. You can find a number of papers about Babel performance and design on Juliusz Chroboczek’s website. Babel already supports routing based on packet loss and latency. Babel is also built to be extended. We’re building a Babel extension to route based on price as well, so at launch, Althea will route packets based on the best combination of price, packet loss, and latency.

Verification is very important in a decentralized context. Without a trusted entity running the entire network, routers cannot count on their neighbors honestly reporting routing metrics. Althea’s verification works quite simply. To learn the quality of all the routes they can service, routers first run a “link quality estimation” with their immediate neighbors. This tests packet loss and latency of the link. This information is gossiped to the network, and routing tables are built by summing the packet loss and latency of every link on a given route.

Althea opens encrypted tunnels with destinations that it is sending traffic to. Nodes forwarding this traffic cannot read it. Babel then runs a link quality estimation over this tunnel. If this turns out to be different than what the neighbor forwarding the traffic claims for the route quality, this neighbor is deemed to be somewhat less reliable. If this happens enough, the neighbor is not used to forward traffic any more. Additionally, new neighbors are not used to forward a lot of traffic until they have proven themselves to be reliable.

RightMesh routes based only on hop count. For hop count, they don’t really need verification, because of the way their payment system works. Their black-box routing protocol chooses routes based on the fewest number of hops, and a smart contract pays all the nodes that can prove they were along a path. The only way to spoof this system would be to generate fake hops in an effort to get a larger proportion of the payment, but then you’d be less likely to be chosen by the routing protocol.

However, hop count is not a very efficient way to route, because you could be choosing to send packets over routes that have few hops, but high latency or packet loss. If RightMesh chooses to use other metrics, they will need to have some way to verify them like Althea does, which their paper makes no mention of. The only other option would be to lean on the proprietary nature of their routing protocol for “security through obscurity”. Perhaps this is what they are counting on. But if there is money at stake, someone will figure out how to crack it, probably sooner rather than later.

Userspace vs kernel implementation

A machine’s networking code must handle every single packet the machine forwards, which can easily exceed thousands every second. The closer the routing operations are to the bare hardware the faster they can go. The Linux kernel has been honed over decades to eke out every last bit of performance when forwarding packets. It’s this code that Althea leans on. Althea uses Babel to populate the Linux kernel’s routing table, and the in-kernel encryption module Wireguard to handle all encryption. Because of this, machines using Althea will usually be able to forward packets at “line rate”- as fast as the transmission medium can handle them.

RightMesh is designed to be run on un-rooted phones, and it uses non-standard proprietary routing and network protocols designed for the “smartphone swarm” scenario they envision. Because of this, all packets forwarded by a RightMesh device will have to be copied out of the kernel, and into “userspace”- an area of the OS where regular apps are allowed to run. All this copying means higher latency, and more memory consumption for lower bandwidth forwarding.

This isn’t necessarily a bad thing though. While a RightMesh device will be limited in speed by the userspace implementation, most phone’s radios can’t handle high speeds anyway. Getting away from the longstanding patterns of traditional networking code will also let them try new techniques possibly better suited to the smartphone networks they are trying to create.

A third factor in this choice is that while Althea will carry any kind of internet traffic from any app, the RightMesh SDK will need to be integrated into every app that wants to use their mesh networking. This could hamper adoption, but might also let them sell more SDK licenses.

Payments

Both Althea and RightMesh depend on either “super nodes” or “exit nodes” running full Ethereum clients and some mesh specialized software to provide peering for transactions to the blockchain and traffic to the internet. What differs is the amount of operations these elevated nodes are tasked to perform and the amount of reporting to them that is required.

Althea’s design is based on the principle of fast payment channels between each hop. Payment for bandwidth is settled before any significant debt can be incurred.. Since the channel payments are so fast and granular, if a node does not pay for service it has received, it can immediately be cut off. This greatly reduces the trust needed in the system.

New nodes, or previously malicious nodes returning under new identities must work their way up a local trust system starting with very small channel balances and frequent transaction costs. By keeping the maximum cut and run gain smaller than the transaction costs incurred by working up the trust system nodes can be either rationally malicious or irrationally malicious at no great harm to the network.

Nodes pay each other in a supply chain model, the same as almost anything else that is bought and sold in our society. Alice pays Bob to forward her packets, then Bob pays the next hop slightly less, keeping some of Alice’s payment as profit.

In RightMesh, all operations of the protocol are on-chain. On any blockchain that exists today, this would make their protocol extremely slow. They introduce the concept of sidechains, which might be able to run their smart contracts more quickly. These sidechains are not specified at all in their paper, but there are several solutions being developed, such as plasma.io and Cosmos. RightMesh will not work until they are finished and deployed.

Instead of Althea’s supply chain model, RightMesh has sort of an auction model. We’re not going to get into all the ins and outs of this process here, but the important part is that every payment interaction in the protocol is mediated by the blockchain or sidechain. This means that the fastest that any routing decisions can happen is many seconds, if not minutes.

Because of this, we’ve had to conclude that RightMesh works on kind of a circuit-switched model. Circuit switching is where a single path or set of paths is chosen for a long lived connection. This is the way the old analog telephone system worked. The revolution of the internet was packet switching. Internet routers are able to decide where to send traffic on a packet by packet basis. No central entity needs to know the path that any given packet takes, and any router can choose to send packets on a different route at any time. Also, the request and response packets making up any given connection do not need to take the same path. It’s this fine-grained flexibility that allows the internet to route around damage and congestion without disruption.

Still, this is probably not an issue for RightMesh’s use case. Supplying internet access from a wifi hotspot to a phone over a few hops does not involve the demands or uptime expectations of a city-level infrastructure network. Their slower, sidechain-mediated model may be sufficient.

Conclusions

We expect to see RightMesh’s protocol change a lot as they try to implement it. They’ll need some kind of route quality verification if they want to route based on anything other than hop count, and they could probably speed up their payments by orders of magnitude with payment channel and state channel concepts. All of Althea’s code is fully open source and permissively licensed, and RightMesh is free to make use of our Babel extensions and our state channel and payment channel libraries.

RightMesh’s marriage to a secret proprietary routing protocol is more concerning. In designing Althea, we found that the networking side of things is much more difficult than managing payments. There are a lot of exploits possible on routing protocols, because almost all routing protocols have been designed for use on friendly networks managed by one entity.

Proprietary network protocols historically not  been successful. The Internet is made up of billions of devices made by tens of thousands of manufacturers and it never could have been built without everyone working together on a standard and open protocol. Especially mesh software, based around cooperation, can not be chained to secrecy. Especially not if RightMesh needs viral growth to fuel the valuation of their token.

Once these issues are overcome, RightMesh will be complementary to Althea. Internet access can be supplied by Althea’s infrastructure-level networks, and distributed to groups of phones with RightMesh.

Check out the video:

Althea's payment channels are key to how it works, but they are also one of the simpler parts to implement and explain. Payment channels allow payments to be made with a minimum of overhead. In the simple case, a payment can be made between two neighbors in the network with only one packet. The catch is that the neighbors need to have deposited money into the a payment channel contract beforehand to "open the channel". However, channels can also be chained together using something called a "hashlock" to allow nodes to transact trustlessly over linked payment channels, without needing to be directly linked by a channel.

There are already several channel implementations for Ethereum. The best-known one is Raiden. We found Raiden to be very monolithic and tied to a large Python offchain codebase. We cannot use Python on our routers. We need to use a language suited to embedded device programming, so we are using Rust (however, this blog post is only about the Solidity contract). Raiden also includes a lot of functionality we don't need, and is very complex in general. For example, they implement their own Merkel tree in Solidity. Maybe the complexity is warranted because it's intended to be a payment channel for every use case. Our application has relatively simple needs and payment channels are a concept that is well understood, so we decided it would be more expedient to use our own simple solution.

There's also Machinomy. Machinomy is pretty clean, but it is a unidirectional channel. This means that money can only ever flow in one direction. You could set up two unidirectional channels, but this is still not as good as a bidirectional channel. To illustrate why, let's say Alice and Bob have both put 100 Ether in a bidirectional channel. If Alice sends Bob 100 Ether, and then Bob sends Alice 100 Ether, the channel will be back in the state it started at and ready to be used some more. If instead there are 2 unidirectional channels, then both participants sending each other 100 Ether over their 2 channels will result in both channels being exhausted. New channels will then need to be created, resulting in more blockchain transaction fees and waiting for confirmations. You could get into some stuff like letting unidirectional channels refill each other, but this is more complex than a bidirectional channel.

We developed the theory underlying our payment channels based on some of Zack Hess's work back in 2015. You can read an explanation of the theory here. For this blog post I'm going to be focusing specifically on the Solidity payment channel contract in the current implementation of Althea.

This code has not been audited, and may have obvious or subtle exploits that we are not aware of. Don't just copy and paste this and put a lot of money in it. If you do see an issue, let us know because we will gladly update this blog post to credit you and discuss the vulnerability.

Imports

pragma solidity ^0.4.11; 
import "./ECVerify.sol"; 
import "zeppelin-solidity/contracts/token/MintableToken.sol"; 

First, we import a couple of other contracts. ./ECVerify.sol is a piece of code written by Alex Beregszaszi (@axic). The type of signatures used by Ethereum normally consist of 3 values, R, S, and V. ECVerify takes a byte array which consists of these three values concatenated. This keeps us from having to pass around 3 extra variables all over the place (the EVM can have issues with too many local variables) and is just cleaner. It also smoothes out some other quirks in the Ethereum signature scheme which are too ridiculous to get into.

MintableToken.sol is an ERC20 token contract provided by Open Zeppelin. It's probably more secure than if we were to try to make our own ERC20 token. None of the stuff in this contract deals with the ERC20 token functionality much, so we could probably plug in lots of different ERC20 variations if we wanted to.

Data structures

contract PaymentChannels is ECVerify, MintableToken {     

struct Channel {         
    bytes32 channelId;         
    address address0;         
    address address1;         
    uint256 totalBalance;

    uint256 balance0;
    uint256 balance1;
    bytes hashlocks;
    uint256 sequenceNumber;

    uint256 settlingPeriodLength;
    bool settlingPeriodStarted;
    uint256 settlingPeriodEnd;
    bool closed;
}

mapping (bytes32 => Channel) public channels;
mapping (bytes32 => bool) seenPreimage;

Now we get into the actual contract. We inherit from the other contracts we imported above, so that we can use their functionalities. We also declare the channel data structure:

• channelID is a unique identifier for the channel.
• address0 and address1 are the two Ethereum addresses that will be transacting over the channel.
• totalBalance is the total amount of money that both participants are putting in the channel. It needs to be the sum of balance0 and balance1
• balance0 and balance1 are the amounts that address0 and address1 are putting in the channel.
• hashlocks is a byte array encoding any hashlocks that are active in the channel. There's parsing logic later in the contract that parses the hashlocks out of the byte array, since passing arrays of structs is not currently supported by Solidity.
• sequenceNumber is incremented with every channel update transaction, allowing this contract to only honor the latest valid update.
• settlingPeriodLength is the amount of time after a channel's settling period has started that must elapse before the channel can be closed and the money can be taken out.
• settlingPeriodStarted: whether the channel's settling period has started.
• closed: whether the channel has been closed.
• settlingPeriodEnd is the block height after which the channel can be closed.

We also declare 2 mappings. channels is for storing channels that have been created with the contract. seenPreimage is a mapping of hashes that records whether the contract has already been shown the preimage of the hash. This is used to implement the hashlocks. Making it global allows the reveal of one preimage to unlock hashlocked funds in several different channels, and uses less storage space.

Modifiers

Next come the modifiers. They are not declared with the modifier keyword, because for some reason it causes the Solidity compiler to complain about too many local variables. However, they have the exact same effect as modifiers when they are declared as functions and called at the top of a function body.

Lifecycle modifiers

These deal with the various lifecycle phases of the channel. This is to prevent functions getting called in phases of the lifecycle they are not allowed.

function channelDoesNotExist (bytes32 _channelId) {
  require(channels[channelId].channelId != channelId); 
}

function channelExists (bytes32 _channelId) {
  require(channels[channelId].channelId == channelId); 
}

function channelSettlingPeriodStarted (bytes32 _channelId) {
  require(channels[_channelId].settlingPeriodStarted); 
}
  
function channelSettlingPeriodNotStarted (bytes32 _channelId) {       
  require(!channels[_channelId].settlingPeriodStarted); 
}

function channelIsNotClosed (bytes32 _channelId) {       
  require(!channels[_channelId].closed); 
}

function channelIsSettled (bytes32 _channelId) {     
  require(
    channels[_channelId].settlingPeriodStarted && // If the settling period has   started         
    block.number >= channels[_channelId].settlingPeriodEnd // And ended     
  ); 
}

function channelIsNotSettled (bytes32 _channelId) {     
  require(!( // Negate the below
    channels[_channelId].settlingPeriodStarted && // If the settling period is started         
    block.number >= channels[_channelId].settlingPeriodEnd // And ended     
  )); 
}

State validity modifiers

These deal with making sure that state updates are valid. For a state update to be accepted, the new balances have to equal the total that was put in the channel, and it has to have the highest sequence number of all the updates that have been submitted.

function balancesEqualTotal (bytes32 channelId, uint256 balance0, uint256 balance1) {     
  require(balance0.add(_balance1) == channels[_channelId].totalBalance); 
}

function sequenceNumberIsHighest (bytes32 channelId, uint256 sequenceNumber) {
  require(_sequenceNumber > channels[_channelId].sequenceNumber); 
}

Signature modifiers

These check that the signatures submitted with function calls are correct. Some functions need signatures from both participants, and some only need signatures from one. Some functions only need one specific signature.

function signedByBoth (bytes32 fingerprint, bytes signature0, bytes signature1, address address0, address address1 ) { 
 require(ecverify(fingerprint, signature0, address0) && ecverify(fingerprint, signature1, _address1) ); }

function signedByOne (bytes32 fingerprint, bytes signature, address address0, address address1 ) {
 require( ecverify(fingerprint, signature, address0) || ecverify(fingerprint, signature, address1) ); 
}

ERC20 balance adjustment

function incrementBalance(address addr, uint value) internal {
  balances[_addr] = balances[addr].add(value); 
}

function decrementBalance(address addr, uint value) internal {
  balances[_addr] = balances[addr].sub(value); 
}

These functions increment and decrement the ERC20 balances. These could be used to create money out of thin air, so they must be handled carefully. They are used to put money into channels and take it out. It might have been possible to give each channel its own address in the ERC20 balances mapping to avoid doing this, but it would have had a lot more complexity and moving parts. The safe math functions (.add and .sub) supplied by the Zeppelin framework prevent any issues like balances going lower than 0.

Creating a channel

function newChannel(     bytes32 _channelId,
address _address0,
address _address1,

uint256 _balance0,
uint256 _balance1,

uint256 _settlingPeriodLength,

bytes _signature0,
bytes _signature1
) {     channelDoesNotExist(channelId);     bytes32 fingerprint = sha3(         "newChannel",         channelId,
_address0,
    _address1,

    _balance0,
    _balance1,

    _settlingPeriodLength
);

signedByBoth(
    fingerprint,
    _signature0,
    _signature1,
    _address0,
    _address1
);

decrementBalance(_address0, _balance0);
decrementBalance(_address1, _balance1);

channels[_channelId] = Channel(
    _channelId,                  // bytes32 channelId;
    _address0,                   // address address0;
    _address1,                   // address address1;
    _balance0.add(_balance1),    // uint256 totalBalance;

    _balance0,                   // uint256 balance0;
    _balance1,                   // uint256 balance1;
    new bytes(0),                // bytes hashlocks
    0,                           // uint256 sequenceNumber;

    _settlingPeriodLength,       // uint256 settlingPeriodLength;
    false,                       // bool settlingPeriodStarted;
    0,                           // uint256 settlingPeriodEnd;
    false                        // bool closed;

);

To create a new channel, someone needs to call newChannel() with the signatures of both participants. We first check that a channel with that channelId does not already exist. Then we check the signatures and attempt to withdraw the money from the accounts of both participants. Finally, we create a record of the channel.

Updating channel state

function updateState(     bytes32 channelId,     uint256 sequenceNumber,
uint256 _balance0,
uint256 _balance1,

bytes _hashlocks,

bytes _signature0,
bytes _signature1
) {     channelExists(channelId);     channelIsNotSettled(channelId);     channelIsNotClosed(channelId);     sequenceNumberIsHighest(channelId, _sequenceNumber);
bytes32 fingerprint = sha3(
    "updateState",
    _channelId,
    _sequenceNumber,
    _balance0,
    _balance1,
    _hashlocks
);

signedByBoth(
    fingerprint,
    _signature0,
    _signature1,
    channels[_channelId].address0,
    channels[_channelId].address1
);

updateStateInternal(
    _channelId,
    _sequenceNumber,

    _balance0,
    _balance1,

    _hashlocks
);
}

function updateStateInternal (     bytes32 channelId,     uint256 sequenceNumber,
uint256 _balance0,
uint256 _balance1,

bytes _hashlocks
)     internal {     channels[channelId].sequenceNumber = sequenceNumber;     channels[channelId].balance0 = balance0;     channels[channelId].balance1 = balance1;     channels[channelId].hashlocks = hashlocks; } `

To pay each other, participants send payments in the form of channel updates that can be passed into this function. We check that the channel exists, that it is not settled, that it is not closed, and that the sequence number of this update is the highest of any received. We then check that all the arguments were signed by both participants. We've broken the actual update logic out into updateStateInternal, because it is called by other functions in the contract. You might notice that we don't check here if the amounts are correct. Someone could submit an update where the balances exceed the total that was put into the channel. We check the balances later, when the channel is closed, after all the hashlocks have been added up.

State update bounties

function updateStateWithBounty(     bytes32 channelId,     uint256 sequenceNumber,
uint256 _balance0,
uint256 _balance1,

bytes _hashlocks,

bytes _signature0,
bytes _signature1,

uint256 _bountyAmount,
bytes _bountySignature
) {     channelSettlingPeriodStarted(_channelId);
bytes32 fingerprint = sha3(
    "updateStateWithBounty",
    _channelId,
    _sequenceNumber,
    _balance0,
    _balance1,
    _hashlocks,
    _signature0,
    _signature1,
    _bountyAmount
);

address bountyPayer = ecrecovery(fingerprint, _bountySignature);

decrementBalance(bountyPayer, _bountyAmount);
incrementBalance(msg.sender, _bountyAmount);

updateState(
    _channelId,
    _sequenceNumber,

    _balance0,
    _balance1,

    _hashlocks,

    _signature0,
    _signature1
);
}

updateStateWithBounty() allows channel participants to incentivize third parties to protect against cheating. Remember, the settling period gives anyone a chance to submit a signed update that has a higher sequence number than the currently accepted update. This prevents dishonest participants from submitting old updates where they have a higher balance. However, someone needs to be online to check whether the settling period has been started with an old update. This function allows the participants to reward someone who is online and stops them from being cheated. The participant simply sends the third party the state update and signatures and their signature on those values and a bounty amount. The third party can now watch the blockchain for an attempt to start the settling period with an old update. If it sees this, it can submit the correct update and collect the bounty.

Submitting preimages

function submitPreimage (     bytes32 hashed,     bytes32 preimage ) {     require(hashed == sha3(preimage));     seenPreimage[_hashed] = true; }
function submitPreimages (     bytes pairs ) {     bytes32 hashed;     bytes32 preimage;
for (uint256 i = 0; i < pairs.length; i += 64) {
    uint256 hashedOffset = i + 32;
    uint256 preimageOffset = i + 64;

    assembly {
        hashed := mload(add(pairs, hashedOffset))
        preimage := mload(add(pairs, preimageOffset))
    }

    submitPreimage(hashed, preimage);
}}
                                     

As you will soon see, Guac's hashlocks follow a simple rule: if anyone has submitted the preimage of the hash, the associated money is unlocked. These functions are for submitting preimages. Preimages are not associated with any one channel. This makes the contract simpler, as well as allowing the reveal of a preimage motivated by money locked in any one particular channel to unlock the money in all the channels participating in that multihop transaction.

Starting the settling period

function startSettlingPeriod (     bytes32 channelId,     bytes signature ) {     channelExists(channelId);     channelSettlingPeriodNotStarted(channelId);
bytes32 fingerprint = sha3(
    "startSettlingPeriod",
    _channelId
);

signedByOne(
    fingerprint,
    _signature,
    channels[_channelId].address0,
    channels[_channelId].address1
);

channels[_channelId].settlingPeriodStarted = true;
channels[_channelId].settlingPeriodEnd = block.number + channels[_channelId].settlingPeriodLength;
}

This only happens when one participant wants to close the channel without the participation of the other. After the settling period is over, the channel can be closed and the money can be taken out. We check that the channel exists and that the settling period has not already been started. We then check for a signature from one of the channel participants, and set settlingPeriodStarted to true and settlingPeriodEnd to the current block plus the settling period length.

Closing the channel

function closeChannel (     bytes32 channelId ) {     channelExists(channelId);     channelIsSettled(channelId);     channelIsNotClosed(channelId);
closeChannelInternal(_channelId);
}

If the channel is settled and has not yet been closed, it can be closed. This checks if the channel is settled and has not already been closed, then calls closeChannelInternal(), which contains the real business logic.

Skipping the settling period

function closeChannelFast (     bytes32 _channelId,
uint256 _sequenceNumber,
uint256 _balance0,
uint256 _balance1,
bytes _hashlocks,

bytes _signature0,
bytes _signature1
) {     channelExists(channelId);     sequenceNumberIsHighest(channelId, _sequenceNumber);

bytes32 fingerprint = sha3(
    "closeChannelFast",
    _channelId,
    _sequenceNumber,
    _balance0,
    _balance1,
    _hashlocks
);

signedByBoth(
    fingerprint,
    _signature0,
    _signature1,
    channels[_channelId].address0,
    channels[_channelId].address1
);

updateStateInternal(
    _channelId,
    _sequenceNumber,
    _balance0,
    _balance1,
    _hashlocks
);

closeChannelInternal(_channelId);
}

If both participants agree to close the channel, they can skip the settling period with closeChannelFast(). They include the final state of the channel, and this function both applies the final update and closes the channel.

Channel closing business logic`js

function closeChannelInternal (     bytes32 _channelId )     internal {     channels[_channelId].closed = true;
int256 adjustment = getHashlockAdjustment(channels[_channelId].hashlocks);

uint256 balance0;
uint256 balance1;
(balance0, balance1) = applyHashlockAdjustment(
    _channelId,
    channels[_channelId].balance0,
    channels[_channelId].balance1,
    adjustment
);

incrementBalance(channels[_channelId].address0, balance0);
incrementBalance(channels[_channelId].address1, balance1);
}

This is the real meat of the payment channel logic. It figures out how much money to transfer back to each participant, while taking hashlocks into account. First, closed is set to true. Then the amount of all the hashlocks is calculated by getHashlockAdjustment(), and applied to the balances by applyHashlockAdjustment(). Then the balances of the participants are incremented by the proper amounts, releasing the money from the channel.

Calculating hashlocks

`js

function getHashlockAdjustment (     bytes _hashlocks )      internal     returns (int256) {     bytes32 hashed;     int256 adjustment;     int256 totalAdjustment;

for (uint256 i = 0; i < _hashlocks.length; i += 64) {
    uint256 hashedOffset = i + 32;
    uint256 adjustmentOffset = i + 64;

    assembly {
        hashed := mload(add(_hashlocks, hashedOffset))
        adjustment := mload(add(_hashlocks, adjustmentOffset))
    }

    if (seenPreimage[hashed]) {
        totalAdjustment += adjustment;
    }
}

return totalAdjustment;
                                          }

getHashlockAdjustment() calculates what effect the hashlocks in the last update have on the balances. We loop through and parse hashes and adjustments out of the hashlock array. There is a totalAdjustment that is added to if the preimage for the hash corresponding to a given hashlock's adjustment has been seen. Putting all the hashlocks in an array like this lets us easily sign them all. This way of doing things does have a limit: you can only submit about 100-120 hashlocks before running out of gas (at least on our testing framework). However if it is necessary, large hubs processing a lot of transactions could have several parallel payment channels.

Applying hashlocks

function applyHashlockAdjustment (     bytes32 channelId,     uint256 currentBalance0,     uint256 currentBalance1,     int256 totalAdjustment )     internal     returns (uint256, uint256) {     uint256 balance0;     uint256 balance1;
if (_totalAdjustment > 0) {
    balance0 = _currentBalance0.add(uint256(_totalAdjustment));
    balance1 = _currentBalance1.sub(uint256(_totalAdjustment));
}

if (_totalAdjustment < 0) {
    balance0 = _currentBalance0.sub(uint256(-_totalAdjustment));
    balance1 = _currentBalance1.add(uint256(-_totalAdjustment));
}

if (_totalAdjustment == 0) {
    balance0 = _currentBalance0;
    balance1 = _currentBalance1;
}

balancesEqualTotal(_channelId, balance0, balance1);
return (balance0, balance1);
}

A hashlock's adjustment, and by extension the summed totalAdjustment of all valid hashlocks, is a signed integer that is added to balance0 and subtracted from balance1. Because of peculiarities in Solidity's number types, it seemed safer and clearer to handle the case of negative and positive adjustments separately. A positive adjustment is straight forward. We cast it to an unsigned integer and do the math. However, when the adjustment is negative casting in Solidity doesn't work correctly. So we invert the sign to ensure that the number is positive and also subtract from balance0 instead of adding and vice versa for balance1. We then check that the balances still equal the total amount that was placed into the channel at the beginning and return the new balances.

Exit node to user node tunnel: The exit node is basically a VPN server, and the physical node uses the VPN for internet access. Users of physical nodes can select an exit node to use and access the internet over a secure tunnel without having to mess with VPN clients on their personal devices. This does not have much to do with incentivized mesh yet, but it forms a base of security and privacy. It can also be useful for people who don't want ISPs selling their browsing history and don't mind setting up a VPN server themselves.

Metering and payments: (under development) The physical node and the exit node both keep track of how much traffic has been sent over the tunnel, and the physical node automatically pays for the traffic. This lets us start to build infrastructure around metering traffic, paying for it, and cutting off service in the case of non-payment. It can also be useful for people who want to set up VPN servers and charge others for the service.

Unverified mesh: Several physical nodes can form a mesh network, which routes traffic to one or more exit nodes, over the best and cheapest paths. Physical nodes meter traffic forwarded for their neighbors, and charge each other for it. This remains vulnerable to low-level attacks on the routing protocol, but will allow us to test the network in a controlled real-world environment, giving users internet access, and receiving payment.

Verified mesh: Physical nodes constantly test the routing metrics advertised by their neighbors, allowing them to verify the quality of the routes, and building a "local reputation" database about their neighbors. This starts to open the network up to secure participation by a wider range of operators, allowing it to grow quickly. Physical nodes still can't be that mobile though, because they need to open channels and verify the routes of other nodes they are paying to forward.

Future evolution...: The "Verified mesh" stage proves out the basic concept. But there's still a lot of work beyond that, some of which is not yet covered even in the white paper:   - Rigorous security testing.   - Multihop channels: Using hashlock techniques, nodes could transfer payments to their neighbors over a network of multihop channels, allowing them to move to a new physical location without waiting for a channel to open to pay for a connection to their new neighbors.   - Transitive reputation: Nodes could also get information on the accuracy of new neighbors from other nodes that they trust on some level. This will also enable mobility.   - On-device user node: Build a stripped down "physical node" app for laptops and mobile devices. This will allow users to pay for traffic and roam without carrying a physical OpenWRT node with them.

How to help

Building this system out requires a large range of skills:

• Linux networking: Obviously a big part of the system.
• Cryptocurrency: We'll be using pretty basic payment channels to start out, so it's not like we need a huge amount of consensus and blockchain wonkery at first. However, the choices we make early on will have big effects, since the space is changing raidly. Multihop channels and transitive reputation will require a bit more expertise later on.
• Frontend design and development: The concept of paying per kilobyte on a cryptocurrency mesh network will be unfamiliar to most people. We'll need intuitive UX/UI design and solid cross-platform frontend code.

If you want to help, or just say hi, introduce yourself in our Matrix chat

There were some very interesting talks. One particularly relevant talk was from Leonardo Macccari, who is part of a team doing some research into community networks for the eu. They've looked into a lot of different networks. what motivates the participants, and in particular, how networks that are supposed to be decentralized can become centralized by influential members owning a lot of the nodes, or dominating discussions.

Another great talk was from Electra Wagenrad, who presented a diy solar charging system for mesh nodes. She estimated that a repeater with 2 directional antennas would require a 50 watt solar panel and a standard lead acid car battery. This could be a great way to set up nodes on hillsides etc with minimal support infrastructure.

I was also able to participate in the testbed setup. I wrote a startup script for Babel and a live updating map showing the strengths of connections between nodes. I learned a lot from the experience and will be able to set up a testbed to refine Althea.

It’s set up in an interesting way. Each router has 2.4 and 5 ghz radios. The 2.4 ghz radios are set up to mesh with 802.11s, to create a management network. Meanwhile, the 5 ghz radios are used to test the routing protocol under consideration. Since 2.4ghz often has a better range than 5ghz, this allows for the testing of topologies that may have poor connectivity in parts. Also, the transmit power of the 5ghz radios was turned down to allow a more realistic topology in a smaller area.

In one on one conversations about Althea, I found that many members of European mesh networks have a pragmatic attitude of “show me the code”. It’s a healthy attitude, and one we agree with. We’ll have a proof of concept soon, which will do most of the payment and tunneling, but without the route verification. This will prove out the concept, while taking a temporary shortcut on some of the more time consuming routing protocol development.

As for the basic idea of incentivized mesh, many of them have thought about it before, but there has never really been a strong attempt at automating payments and verification the way we are doing. Guifi.net in Catalonia has a system where for-profit ISPs are able to use their network and contribute back to it in proportion to how much they are using. However, this is all manually calculated and worked out by humans.

I also spoke with Axel, the creator of BMX7, and Linus, who works on Batman. We discussed the technical details of our verified routes. It didn’t seem like they had really considered the idea of verifying the quality metric of routes much before, but Axel has put several mechanisms into BMX7 to stop attacks aimed at simply disrupting the network. We will likely need to add similar mechanisms to Babel, although Babel is working on signed messages which may help prevent a lot of these attacks.

All in all, it was a good experience. They’ve perfected their testbed setup, and the knowledge of how it’s put together will be very helpful for us in the coming months. As for the likelihood of existing community networks being early adopters, I think that they may be a little more conservative because they already have networks up that work with their technical and social structure. They’d probably want to see something pretty well developed before risking their existing technical and social capital.

Also, some members of the community view the inclusion of a monetary incentive with suspicion for political and social reasons. Some of them might have to see Althea increasing competition, improving service, and lowering prices before giving the idea a chance.

Pay for forward

• A (secured) routing protocol propagates price along with the route quality metric. The routing protocol makes routing decisions based on the price and the metric.It is secured by running speedtests to destinations on the network and seeing how they match up to the advertised metrics. Using a "circular ack" speedtests can be reused many times by all the nodes on a route to a given destination. This also works with asymmetric routes.
• Nodes tally up what they are owed their neighbors according to which destinations the neighbors are sending packets to. Payments are settled with payment channels. If payments are not made, a node can stop forwarding a neighbor's packets using tunnels. On top of the pay-for-forward system, there is a gateway discovery and payment system. Somehow end-user nodes discover "gateway" nodes, nodes with a connection to the internet. Gateway nodes charge a certain amount for the internet connection. When a gateway is chosen by an end-user node, a payment channel is opened with that gateway. The gateway is paid for the amount of data used from the internet, as well as the amount that it cost the gateway to pay-for-forward to send the packets back to the end-user node. Gateways would need to be speedtested. Speedtests could be done between the end-user node and arbitrary servers on the internet.

Pay for internet

• Nodes pay each other for access to the internet.
• Some secure source of routes is neccesary, but it remains completely separate and unspecified by this scheme.
• A node which is closer to the internet will be paid to both send and receive packets on behalf of a node which is further away (upload and download connection).
• Nodes must discover and verify the quality of the internet access provided by their neighbors. This somewhat mirrors the quality metric discovery and verification in a secure routing protocol. To verify the upload quality of a neighbor, a node sends traffic to a speedtest server on the internet. To verify the download quality of a neighbor, a node has a speedtest server on the internet send it traffic. In most routing protocols, there will only be one upload neighbor and one download neighbor at once (and they will often be the same neighbor). However, if there are multiple upload or download neighbors, it would be very difficult to tell which one is responsible for the quality of a route.
• Nodes could tally up what they are owed by their downstream neighbors according to how much data they have sent or received on behalf of a given downstream neighbor. Payments are settled with payment channels. If payments are not made, a node can stop forwarding packets to or from a neighbor using tunnels.
• Instead of announcing an all-or nothing price for access, nodes could instead prioritize traffic to and from neighbors according to how much each neighbor was paying at a given time. This might allow a natural range of quality/price options. It makes access discovery and verification harder. How do you know how much a neighbor's quality will improve if you pay it more?

• Establishes authenticated tunnels with each of its neighbors. This is so that nodes can reliably tell each others traffic apart. Simply relying on MAC addresses would allow any node to spoof the traffic of another node to get free service.
• Blocks packets that have not come over one of the authenticated tunnels from being forwarded.
• Prioritizes traffic from some neighbors more highly than others. This will ultimately be linked to payment, but for now I'll just pass it in manually.
• Allows nodes to choose what proportion of their total bandwidth they will share/sell.

In the previous post, I wrote about doing this with shell scripts. These were fast and easy to write, and a good way to test the technique. Now I am working on doing the same thing with a robust, configurable, and well-tested piece of software that does not rely on manual tweaking.

I've been writing it in Go. The first thing that needs to be done is discovery. Neighbors must be able to establish an initial contact with one another. I've been doing this by sending UDP packets to a specific port on the IPv6 multicast address. Nodes send "hello" messages on this port. The hello messages contain the node's public key and "control address", where messages can be sent to communicate with the node.

Scrooge hello message

This is sent on the multicast port to discover neighbors. It can also be sent directly to a neighbor to notify them of an update of the control address.

scrooge_hello <publicKey> <control address> <seq num> <signature>

• PublicKey: base64 encoded ed25519 public key. This is used by neighbors to identify each other and sign messages, including the scrooge_hello message.
• Control address: The IP address and port where one can send out of band control information.
• Sequence number: Incremented with each hello to prevent playback attacks
• Signature: The signature of the publicKey over the fields of this message, concatenated as byte strings with no spaces (we may want to tweak this?)

When a node receives one of these messages:  - It first checks the signature, and adds a record of this Neighbor if it does not already exist (neighbors are identified by public key).  - It checks the SeqNum to prevent replay attack. - It updates the ControlAddress of the neighbor with the given PublicKey. - It sends a scrooge_hello_confirm message to the neighbor’s control address

Scrooge hello confirm message

scrooge_hello_confirm <publicKey> <control address> <seq num> <signature>

When a node receives one of these messages:  - It first checks the signature, and adds a record of this Neighbor if it does not already exist (neighbors are identified by public key).  - It checks the SeqNum to prevent replay attack. - It updates the ControlAddress of the neighbor with the given PublicKey. - It may start a tunnel and send a scrooge tunnel message as described below.

Scrooge tunnel message

Sometimes a node wants to establish a tunnel with one of its neighbor nodes. Maybe it has just received a hello_confirm from this neighbor after it has broadcasted a hello, or maybe it needs to refresh the tunnel for some reason.

• It first stops and removes any existing tunnel with the neighbor.
• It then starts a new tunnel on an available port and sends the message.

scrooge_tunnel <publicKey> <tunnel publicKey> <tunnel endpoint> <seq num> <signature>

When a node receives this message, - It adds the tunnel publicKey and endpoint to the tunnel record for that node and starts a tunnel listening on an available port. - It then sends a scrooge_tunnel_confirm message back.

Scrooge tunnel confirm message

scrooge_tunnel_confirm <publicKey> <tunnel publicKey> <tunnel endpoint> <seq num> <signature>

This is the same as the scrooge_tunnel message, except that when a node receives it, it does not send a message back. This is to stop an infinite loop of scrooge_tunnel messages from occurring.

The parsing and handling of these messages is all unit tested, and I'm currently working on integration tests, running in CORE.

In the original design of Althea, in the whitepaper, I call for verification of routes. That is, once a node has made a connection to another node that it cares about, it checks whether the route quality information propagated with the routing protocol is correct. This consists of the node doing some kind of route quality or throughput test with the destination node and seeing if it matches up to what was propagated.

If the propagated information is incorrect, the node corrects its own routing information, and tells the node that gave it the bad route about the inaccuracy. Once the neighbor finds out about the inaccuracy, it could do its own quality test, and tell its neighbors about the results and so on. But having each node along a route doing a quality test to verify the route could be slow and result in a lot of overhead.

Justin Kilpatrick, who is working on a similar project called Hocnet, came up with an interesting idea: reuse the results of a quality test along a route.

As a simple example, let's say that the quality metric that is being used is percent packet loss. Alice is sending traffic to a Zack over the network. Zack keeps track of how many packets he receives over a certain time period, signs this, and sends it back. Now Alice can compare this information with her own statistics about how many packets she has sent out during the same time period to derive a percent packet loss metric.

Let's say that the numbers don't match up, and the route is losing more packets than advertised.

In Althea, Alice would just correct her own routes and notify Bob, the neighbor that gave her the route. Bob would then have to communicate with the destination himself to find out what the real quality is.

In Hocnet, Alice passes Zack's signed statement about how many packets he has received from her along to Bob. Now Bob compares Zack's statement just like Alice did to find out his own quality to Zack.

One additional requirement in this kind of scheme is that the quality test be done on normal traffic. Otherwise, nodes along the route could simply prioritize traffic that they recognize as part of a quality test, while dropping normal traffic.

Possible types of quality test:

Packet loss percentage:

Functionality: Nodes find out how many packets are lost over links to their neighbors. When they get a route update message for a given destination, they multiply the total route packet loss by the packet loss of the link they received it on and forward it onwards.

Verification: The product of the packet loss of every link in a route should equal the packet loss of the whole route. The destination sends a signed message with the number of packets received from a source during a certain time period. Nodes along the route can compare this with how many packets from that source they have forwarded to the destination during this time period to find the route's total packet loss. Babel and Batman IV use this type of metric.

Throughput:

Functionality: Nodes find the maximum throughput of links to their neighbors. When they get a route update message for a given destination, they set the total route throughput to equal the throughput of the link they received the message on, if the throughput of the link is lower.

Verification: The lowest throughput link along the route should equal the maximum throughput of the route as a whole. The destination sends a signed message with the number of bytes received from a source during a certain time period. Nodes along the route can compare this to the number of bytes that they forwarded from that source. An important caveat here is that the route must be saturated to the maximum throughput to get an accurate estimate. This could waste bandwidth and also tip off nodes along the route that a quality test is happening. Batman V uses this type of metric.

Latency:

Functionality: Nodes measure the average latency of links to their neighbors. When they get a route update message for a given destination, they add the latency of the link they received it on to the total route latency before forwarding it onwards.

Verification: The sum of the latencies of all the links on a route should equal the latency of the route as a whole. The destination sends a signed message with the hashes of various packets and their time of receipt. Nodes on the route compare this time of receipt with the time that they received each packet.

• Establishes authenticated tunnels with each of its neighbors. This is so that nodes can reliably tell each others traffic apart. Simply relying on MAC addresses would allow any node to spoof the traffic of another node to get free service.
• Blocks packets that have not come over one of the authenticated tunnels from being forwarded.
• Prioritizes traffic from some neighbors more highly than others. This will ultimately be linked to payment, but for now I'll just pass it in manually.
• Allows nodes to choose what proportion of their total bandwidth they will share/sell.

I've been experimenting in CORE with some shell scripts to make this stuff happen. I'm going to write the actual software in Go, but shell is great for prototyping. You can call other programs from within the language and shell's jankiness and disposable feel keeps you from getting too fixated on the minor details.

Tunnels

I'm using fastd for tunnels because it is easy to use and has a lot of options for customization. It's in userspace, which means that each packet will be copied out of kernel memory and back in several times. This greatly limits bandwidth, so I'll switch to something more efficient like Wireguard at some point.

Right now I'm using this shell script to set up each peer:

`shell

usage: addpeer <peernumber> <peerip> <peer_iface>

mkdir -p peers

peernumber=$1 peerip=$2 peer_iface=$3

touch peers/peer$peer_number

echo '   key "'$(cat /home/jehan2/host/fastd-test/publickey$peernumber)'";   interface "%n";   remote "'$peerip'" port 1234; ' > peers/peer$peer_number `

ip route add $peerip dev $peeriface

Fastd uses a peers/ folder to keep track of its peers. Each peer has a file that contains its public key, address, and other information. Here I'm creating the file for a given peer. I'm copying the public key from a central folder.

To set up a network that looks like 1--2--3, I run these commands:

On node 1:

$ addpeer 2 10.0.0.2 eth0

On node 2:

$ addpeer 1 10.0.0.1 eth0 $ addpeer 3 10.0.0.3 eth0

On node 3:

$ addpeer 2 10.0.0.2 eth0

Then I run this script on each node to get things started:

`shell

usage: start <mynumber> <mytunnel_ip>

touch fastd.conf echo \ '   bind any:1234; # UDP Port 1234 auf allen Interfaces   mode multitap;   method "xsalsa20-poly1305"; # Verschlüsselungsalgorithmus festlegen   mtu 1426;   secret "'$(cat /home/jehan2/host/fastd-test/privatekey$1)'";

 include peers from "peers";

 on up "     ip link set up $INTERFACE     ip addr add '$2' dev $INTERFACE     echo interface $INTERFACE | nc ::1 8481   "; ' > fastd.conf

fastd -d -c fastd.conf `

This sets up each node's private key and other fastd settings, including the tunnel_ip. The tunnel_ip is different from the IP that the node has on the physical network. It is only accessible over the authenticated tunnels. I might actually be able to just use the node's normal IP for this, but having it be a different address is a good sanity check.

I also have to set up routes on each node manually. This will be done by the routing protocol, but for these tests I am doing it manually to eliminate any uncertainty.

Firewall

Nodes do not want to forward any packet that has not come over one of the authenticated tunnels, because this packet has not been paid for. This is actually pretty easy to set up with nftables, once you understand the syntax and how it operates.

`shell

usage: addrules <peer_number> ...

nft flush ruleset nft add table althea nft add chain althea fwd "{ type filter hook forward priority 0; policy drop; }"

for iif in "$@" do   for oif in "$@"   do     if [ $iif != $oif ]; then       nft add rule althea fwd meta iifname "peer$iif" oifname "peer$oif" mark set $iif$oif accept     fi   done done

nft list ruleset -a `

This script sets up a table and a chain (these are both just containers for organizing the firewall rules). The chain is set up with a default policy of just blocking everything, and it is attached to the forward hook which means that it applies to packets that are about to be forwarded by the system. By itself, this chain will prevent the system from forwarding any packets. In the loop the script makes a rule to accept packets for every pair of nodes, and adds a unique mark to them. This will allow us to identify which nodes packets are coming from and going to in the next step.

Running the script

$ sh addrules.sh 2 3 4

results in this ruleset:

table ip althea { chain fwd { type filter hook forward priority 0; policy drop; iifname "peer2" oifname "peer3" mark set 0x00000017 accept   iifname "peer2" oifname "peer4" mark set 0x00000018 accept   iifname "peer3" oifname "peer2" mark set 0x00000020 accept   iifname "peer3" oifname "peer4" mark set 0x00000022 accept   iifname "peer4" oifname "peer2" mark set 0x0000002a accept   iifname "peer4" oifname "peer3" mark set 0x0000002b accept   } }

Traffic Shaping

Now it is time to prioritize traffic depending on how much nodes are paying. I'm using tc, which is a tool in linux for this purpose. Traffic shaping is a very subtle process and I don't fully understand all the ins and outs yet. Also, tc's syntax is frankly quite cumbersome. Tc lets you set up qdiscs- (short for queueing discipline). There are also filters which can put packets into different classes inside the qdisc depending on different criteria. The qdisc then enqueues packets into the different classes, and either dequeues them to the network or drops them to accomplish its traffic shaping goals.

I'll start with a diagram to explain the shaping setup that I have made, because the tc scripts are hard to follow.

qdisc htb- limits all traffic to 10mbit/s  | qdisc drr- splits traffic into classes to be prioritized, based on the marks we added to packets with nftables  |                            | filter fw 23                 filter fw 24  |                            | class drr quantum 1400        class drr quantum 700       ...etc  |                            | qdisc fq_codel               qdisc fq_codel

First, the htb qdisc limits all traffic to 10mbit/s. This will allow nodes to choose how much of their bandwidth they wish to sell, and it is necessary for the drr qdisc to function (I'm not entirely sure why).

The drr qdisc has filters attached which put the traffic into its child classes based on the marks we attached to the packets in the firewall step above. Drr operates in a round robin fashion and basically allows each class to transmit "quantum" bytes each round. A class with quantum 1400 should be able to transmit twice as much data as one with quantum 700.

The fq_codel qdisc drops packets that hang around in each queue for too long. It also fairly queues traffic within each class so that, for instance, a large download won't slow down other traffic in the class too much. Without this packets will stay in the queues forever resulting in the system slowing to a crawl.

In the above setup, traffic going from peer 2 to peer 3 will have twice as much bandwidth as traffic going from peer 2 to peer 4.

Here are the scripts:

`shell

usage: setup <network interface>

dev=$1

Remove root qdisc (clean slate)

tc qdisc del dev $dev root

Add root shaping qdisc and class

tc qdisc add dev $dev root handle 1: htb tc class add dev $dev parent 1: classid 1:1 htb rate 10mbit ceil 10mbit

Add root drr qdisc

tc qdisc add dev $dev handle 2: parent 1:1 drr

Add default class

tc class add dev $dev parent 2: classid 2:1 drr

Send traffic to default class

tc filter add dev $dev parent 1: protocol all prio 2 u32 match u32 0 0 classid 1:1 tc filter add dev $dev parent 2: protocol all prio 2 u32 match u32 0 0 classid 2:1

tc -s class show dev $dev tc -s filter show dev $dev `

This sets things up and adds the htb qdisc and the root drr qdisc.

`shell

usage: addqueue <network interface> <firewall mark> <quantum>

dev=$1 handle=$2 quantum=$3

Get traffic with a certain fw mark and direct it to the corresponding DRR class

tc filter add dev $dev parent 2: protocol ip prio 1 handle $handle fw classid 2:$handle tc class add dev $dev parent 2: classid 2:$handle drr quantum $quantum

This acts to actually drop traffic, making DRR take effect

tc qdisc add dev $dev parent 2:$handle fq_codel

tc -s class show dev $dev tc -s filter show dev $dev `

I run this for every child class of the drr qdisc that I want to create. This will typically be once for every rule I set up in the firewall script above.

Using this setup, I am able to prioritize traffic coming from and going to a node's different peers. The next step, which I am working on now, is to automate this all. I'm writing that code in Go.

Discuss this post on Reddit

What I was trying to do was this: Nodes propagate pricing information to one another using a distance vector routing protocol (Babel), and then pay neighbors for data sent. They also pay the exit node (a node with internet access) of their choosing, who in turn pays to have the data sent back to them. They make sure that quality along the way is consistent with what is advertised by using speed tests. The service is binary: nodes pay each other an agreed-upon rate for connectivity, or they don't get service.

I'm now working on a new scheme that simplifies and tweaks some stuff:

• I'm not addressing routing at all. The ideal routing configuration for any given network depends on a lot of factors, and mandating a certain routing protocol will limit the usefulness in different scenarios. Anything from manually entered routes, to ad-hoc protocols, to centrally administered SDN could work.
• I no longer am making any distinction between normal nodes and exit servers. This distinction will still exist, of course, but Althea won't be involved with it.
• Nodes will pay for both upstream and downstream service, instead of the pay-for-forward model I was using before.

What's left is a piece of software that looks like this:

• Nodes set up authenticated tunnels with all their neighbors. This is so that they can tell for sure that a specific packet was routed by a specific neighbor. These are TAP tunnels like the ones used for VPNs. Two promising options are fastd and wireguard
• Nodes then prioritize traffic to and from their neighbors, depending on how much each neighbor is paying. Linux's tc tool should be able to do this. This way, purchasing service is a smooth continuum, instead of a binary state.

This is very simple, and will need to work together with a few other components to make a useful system.

• Routing is still necessary, and it still needs to be secure. Babel, and most other routing protocols will allow nodes to cheat and direct more traffic towards themselves than they should have. There are some ideas in the direction of securing distance vector protocols such as SEAD. Another option is using centrally determined routes or SDN. It might be easier to monitor and prevent cheating with a centralized system.
• Just because nodes can choose their desired level of prioritization and compete for service with the nodes around them, doesn't mean they know what prices to pay. The simplest option is to have the pricing manually controlled, but this will be too much of an annoyance for most people. An automatic pricing system is probably a necessity for wide adoption, but I am leaving it for later.

Discuss this post on Reddit

Alice and Bob wish to exchange signatures. If Bob sends his first, Alice could simply refuse to send hers and walk away. If Alice sends first, the situation is reversed.

Here's a good overview from a question I asked on a Q&A site.

This applies very directly to state channels. Let's review how a state channel works:

Alice and Bob agree on some state, give it an incrementing sequence number, and sign it. Either Alice or Bob can now prove that the other agreed to that state, and that it is the most recent state that they have agreed to. If either one tries to falsely claim that an older state is the most recent state, the other can show a signed state with a higher sequence number and disprove the claim.

Most current state channel implementations also include the concept of a challenge period. This is a period of time after which it is not possible to disprove an older update by submitting a newer one. This gives a distinct cutoff point after which actions can be taken on the state.

This mechanism has found use in payment channels, which allow Alice and Bob to keep a running tally of how much they owe each other, updating it to make payments with extremely low overhead. This tally is backed up by money escrowed by one or both of them on a blockchain or with a trusted third party. When either one wants to pull the money out, they submit the most recent state. After the challenge period ends, the money is released.

In theory it can also be used for a wide range of other interactions including pong.

Example

The free option problem is just the fair exchange problem in the context of state channels. It's called the free option problem because it first presented itself in the use of state channels for trading assets against each other. Let's say a state channel is being used by Alice and Bob to trade silver against gold. The state records Alice's balance of silver and gold, and Bob's balance of silver and gold. This is backed up by bars of metal stored with an escrow service. When Alice wishes to trade some silver from some gold, she creates a new state with her balance of silver decreased and her balance of gold increased, and Bob's balances vice-versa. She increments the sequence number, signs it, and sends it to Bob.

If Bob refuses to send his signature back, he now has a free option. If he wants to, he can submit the state that Alice just sent him and receive more silver, or submit the previous state and receive more gold. He might choose differently based on how the market moves. Alice has given him a free option.

The free option problem also has another effect. Let's call it an "unjust punishment". Some state channel implementations have a mechanism where someone attempting to close a channel must submit a deposit along with their closing state. If a later state overrides it, the deposit is taken as punishment. This is meant to dissuade people from dishonestly submitting old states. In the example above, if Alice wants to close the channel, she must submit the latest state that she has. However, Bob has a later state with a higher sequence number. If Alice submits her latest state, Bob can submit his later state and take her deposit.

Mitigations

There are at least 3 mitigations of the free option problem. They all basically consist of eliminating a single point in time where one party has a fully valid state and the other does not.

Incremental transfer amount

This applies mostly to channels recording balances of assets. More broadly, it works for channels where a state update can be broken into many smaller updates to achieve an equivalent effect. For instance, a trade of 100 grams of silver for 10 grams of gold can be broken into 100 trades of 1 gram of silver for 0.1 gram of gold. In the example above, if Alice did this she would only ever be giving Bob a much smaller free option. However, this only really works for divisible numerical transfers, and it does nothing to protect against an unjust punishment.

Incremental transfer validity time

A solution used by Gnosis is to make state updates that expire after some time. Alice sends Bob an update transferring the full amount that expires very soon. When Bob has sent her a signature on that update, she sends an update that is valid for longer. This way, Alice and Bob are giving each other free options that are not useful for very long. This is especially well suited for exchanges, where the value of a free option is determined by how much the market has moved. It is also a partial solution to the unjust punishment problem discussed above. If Alice wants to close the channel, but Bob has a later update, she can just wait until Bob's update is no longer valid.

Incremental transfer probability

This is an interesting solution that I came up with, which to my knowledge is not implemented anywhere. Alice and Bob each have a number of separate keypairs that they sign updates with (let's say 100 for now). Alice sends Bob an update that is signed by 1 of the keypairs. When Bob has responded with the signature from 1 of his keypairs, Alice sends a signature from her next keypair. This continues until the update has 100 signatures from each Alice and Bob. When one of them wants to close the channel, the blockchain or escrow service chooses a random number. This is used to derive the number of signatures required to consider the update valid. For instance, if the escrow chooses 0.34 as the random number, 34 signatures from Alice and Bob are required. Of course, if the process has been followed correctly and completed, the chance of validity is always 100%.

If one of them breaks the process off early, they will only have a 1% higher chance of having a valid state update. This obviously is not great if one party has a much more at stake or a much lower tolerance for risk than the other. I think this mechanism could also be used to mitigate the unjust punishment problem, if there is also a punishment for updates that fail validation for not having enough signatures.

First, let's look at the alternatives that I evaluated before settling on CORE.

Simple container network

The first thing I tried was just using some containers in a network, and then using the Linux tools iptables and/or tc to selectively drop and delay packets between pairs of nodes to simulate a lossy radio network. This would obviously not uncover complex real-world network dynamics, but no simulation will do this perfectly.

The advantage of this approach could be simplicity (maybe). It's also a good way to improve my chops with the Linux network utilities.

One example of this is mlc, written by Axel Neuman, the creator of the BMX6 routing protocol. Unfortunately, this package requires a lot of manual setup, and hasn't been updated in a few years. It's still something I will get familiar with, but I may evaluate some other more polished tools first. This technique starts to lose its simplicity the further you try to take it.

Mininet

Mininet is a more polished tool that sees a lot of use in the software defined networking (SDN) community. It can create networks of virtual computers connected to SDN-controlled OpenFlow switches.

Mininet looks nice, because it is relatively lightweight, with all functionality programmable through a REPL or a nice Python API. The API is pretty well documented and there are many examples.

I'm intending to revisit Mininet, but to use it effectively I think that I would need to get pretty familiar with OpenFlow and the concepts of SDN, and then use it to simulate a mesh network. No doubt that these are interesting and valuable skills, but maybe it would be better to spend time learning something actually targeted at the wireless mesh use-case.

Mininet-wifi

Mininet wifi adds WLAN simulation capability to mininet, with what looks like some pretty slick packet loss simulation stuff. However, it appears to be geared towards simulation of access points in a centrally-configured SDN network, not use in an ad-hoc mesh network.

CORE

CORE intimidated me with complexity at first, in addition it is not as well documented as Mininet. However, after getting it started (with the help of a very convenient VM), I found that it seems to be a capable toolkit. You get a window where you can drop nodes and link them together to form connections. Or, you can create a WLAN network and allow it to calculate connectivity based on the node's positions on the screen. CORE also integrates EMANE, which is a much more full-featured wifi simulation suite. Finally, CORE can also import mobility scripts, which allow you to automate the movement of nodes on the canvas to see how well routing protocols do with mobility.

IMUNES

CORE is built on top of IMUNES, which is another network simulator with a visual interface. It looks good, and is under more active development, but CORE seems simpler to get going for a mesh simulation.

Using CORE

CORE has a pre-built VM image available to make it easy to get going. I installed VirtualBox to run the image, and started it up.

Clicking on the icon, you're greeted with a blank canvas with some toolbars to the side. Click on the toolbar button with the router symbol (network-layer virtual nodes), select the "MDR" type, and place a few of them on the canvas. Then click on the button below it (link-layer nodes) and place a WLAN node, which looks like a little cloud. Right clicking the WLAN node brings up a menu that has the option "Link to all routers". Clicking this links all nodes to the WLAN network and gives them IP addresses.

Clicking the play button starts the simulation. If the nodes are close enough together, green lines will form, representing the connections between them. If you double click on one of the nodes, it opens up a terminal. From here, you can try to ping other nodes. To get a quick overview of connection health, I like to use the command ping -f <ip address>. The -f flag formats the output in a special way. Each time it sends a ping, it prints a dot. Each time it receives a response, it prints a backspace, deleting the prior dot. This effectively means that dots correspond to timed-out pings. If the line of dots grows rapidly, a lot of packets are being dropped. If it grows slowly or not at all, most packets are arriving at their destination.

You can move nodes around while the ping command is running. If you move two nodes far enough away from each other that the green line between them disappears, all packets will suddenly start being dropped. If the green line is there, all packets are received. This is obviously not a very sophisticated simulation. This will not work for testing routing protocols, which must be able to find the best route among links of varying reliability.

Luckily the CORE emulator includes EMANE, which simulates the lower-level networking stack (tcp, wifi, and below), and the physical layer (radio waves). With EMANE, moving nodes further away from each other increases the drop rate and the latency. I didn't necessarily need the level of physical accuracy that it provides, but hey, it's pretty cool. Right click on the WLAN node, and select one of the EMANE models.

Ok, let's run Babel on our nodes. Clone the Babel repo, and make and install it etc. Now double click on one of the router nodes to bring up the terminal. You should be able to type something like /home/core/babeld/babeld eth0 -d 1 to start babel. The -d 1 flag puts it into debug mode so that it prints logs to the terminal, and we can verify that it is running.

Next, we need to set up a custom service to make Babel start automatically on new nodes. Go to ~/.core/myservices and read README.txt. I'm not going to duplicate what it says there, but my Babel service looks like this:

babel.py

import os

from core.service import CoreService, addservice
from core.misc.ipaddr import IPv4Prefix, IPv6Prefix

class BabelMod(CoreService):
    # a unique name is required, without spaces
    _name = "BabelMod"
    # you can create your own group here
    _group = "Routing"
    # this controls the starting order vs other enabled services
    _startindex = 50
    # list of startup commands, also may be generated during startup
    _startup = ('sh /home/core/myservices/babel.sh',)
    # list of shutdown commands
    _shutdown = ('rm log')

# this line is required to add the above class to the list of available services
addservice(BabelMod)

babel.sh

#!/bin/sh

touch log
/home/core/babeld/babeld eth0 -d 1 > log

Restart CORE to install the new service.

Let's make a new node type which will only route using your new Babel service. First, click on the network-layer nodes button and select "edit node types". In this window, there's an "add new node type" button under the list of existing node types. I called mine "babel", and gave it a purple icon. Click on the "Services" button and make sure that only the "IPForward", and the new "BabelMod" services are selected.

Now you should be able to place nodes on the canvas, and have their babel instances start talking to each other. Since we piped the output of Babel to the log file, we can get some insight into its operation. Double click on a node to open its terminal and type tail -f log to see what Babel is logging.

Now that we have Babel running, we can try to see if we can get it to do some routing. Place 4 babel nodes on the canvas, 2 far apart from each other and 2 in the middle. Make one of the middle nodes a much better route than the other (closer to the 2 end nodes). Wait a few seconds, and you will see that the end nodes have routes to each other going through the better-placed middle node (you can check with Widgets->Observer Widgets->IPv4 Routes). You can also ping from one end node to the other to monitor packet loss.

Now switch the positions of the middle nodes. You will see the packet loss jump, as the end nodes attempt to route packets through the middle node which is now in a bad position. After around 30 seconds, you will see throughput improve again as the new route is learned. If you check the routing tables of the end nodes, you will see that they contain the new and better route.

Ameen is actively researching state channels and was looking for project ideas. I suggested pong, as it is a very fast paced game that really emphasizes the scalability and speed of state channels. Playing pong on a state channel is somehow much more impressive than tic tac toe or chess. Thinking about state channels in terms of pong also loosened up my thinking around channels in general. We talked with Zaki for about an hour to find a good way to do it.

Alice and Bob send frames back and forth, both signing each valid frame. For example:

Alice -> Bob Frame:

Sequence: 88     
Ball: 11,23     
Alice's paddle: 15     
Bob's paddle: 3

Bob -> Alice Frame:

Sequence: 89 // Bump the sequence number     
Ball: 12,24 // The ball has moved according to pong "physics"     
Alice's paddle: 15     
Bob's paddle: 4 // Bob has moved his paddle 

Alice will sign the frame that Bob has given her if it is valid, i.e. if the sequence number is correctly incremented and if the ball has obeyed the laws of pong physics. The frame also contains an update of Bob's paddle position.

They set up a contract on the blockchain that also understands the state transition from one frame to another. If given a frame signed by both Alice and Bob, the contract can of course establish that it is valid. The contract can also establish that a frame signed by only one of the participants is valid, given the frame before it signed by both. Also, as is the norm in state channels, frames with a higher sequence number override those with a lower sequence number. There is a challenge period of some length as well. After a frame is submitted, the contract waits for this amount of time before taking action on it (i.e. paying out some money to the winner).

In addition to frames, the contract accepts a notice of forfeit, signed by only one of the participants. For instance, Alice can claim "Bob forfeited". This can be overriden by a frame of any sequence number. If Bob does not post any valid frames during the challenge period, he forfeits the game.

Let's say that Bob sees that Alice is about to win, and he would like to end the game without forfeiting the money that he has locked up in the contract as a bet or whatever. I will prove that it is not possible for Bob to do this.

If Bob simply disconnects and stops signing frames, Alice can send a notice of forfeiture to the contract. If Bob doesn't send a valid frame before the hold period is up, Alice gets the money.

If Bob does send a valid frame, it's now up to Alice to send the next frame before the challenge period is over, otherwise she has forfeited. Alice and Bob could go back and forth like this until the end of the game, but this would be a very slow game, and very expensive because they would have to pay gas each time they submitted a frame.

If Alice sends a notice of forfeiture and then refuses to sign any more frames, Bob can submit a frame signed only by him, which is a valid transition from the frame signed by both him and Alice. Now it's up to Alice to send the next valid frame.

It's in both of their interests to play honestly and keep all frames except for the last one off the chain, but the slow and inefficient frame-by-frame process is always there as a fallback to keep the players honest.

The essay adopts a tone that has been common in the week since the DAO hack, a conviction that Turing-complete languages are doomed to fail, and that anyone who would use them for anything is stupid. I don't think this is the case. There's a huge amount of critical code in the world which is Turing-complete, and things are OK. Do software vulnerabilities exist? Yes. But they do not stop us writing complex and sensitive systems.

Tony's piece makes the distinction that the really dangerous thing is to have Turing-complete input languages. In my reading of some of the langsec material that he cites, this distinction is made because so many programs are written with the mentality that their input is simple and is not worth treating as a language. A lot of bugs have arisen because of a program acting more like a parser, and its input acting more like a language than the programmer suspected. I don't feel that Ethereum's EVM bytecode qualifies as an input language under this criteria. It's intended to be a language, and it's intended to be Turing complete. It has come, and continues to come under the scrutiny appropriate to such a role.

The idea is that Turing complete languages are undecidable, and thus, that it is impossible to prove that they are secure. Indeed, one cannot prove that a Turing complete program will ever stop running (the halting problem). This doesn't stop us from using Turing complete code in critical applications. It's impossible to formally verify that your airplane isn't going to crash into the ocean, but that doesn't stop you getting on a plane. Because of the dangers of unexpected behavior, we make sure to test carefully, and use programming patterns that prevent bugs.

Turing complete EVM bytecode has all the same vulnerabilities of the software surrounding us, flying airplanes, running bank servers, and even driving cars. In the case of self-driving cars, neural networks perform a lot of their logic. Neural networks are far beyond any kind of static analysis or formal proof. The only way to verify that they are correct (and indeed the only way to program them) is to test them many times with many different inputs and make sure that they behave as expected. This doesn't stop us from making use of them.

The vulnerability that killed the DAO was very simple in nature. When you send money to an Ethereum contract, it has the option to run some code upon receiving the money. The mistake made by the authors of the DAO was to debit the account of the sender after sending the money. The attacker's contract called the DAO's sending function again, triggering another send before debiting the attackers account for the first send. This vulnberability was actually quite similar to the vulnerabilities that can exist in databases that do not implement transactions (like MonogDB). These are bad vulnerabilities, but they are well understood and it is easy to mitigate them with mechanisms such as 2 phase commits, mutexes, etc.

While the vulnerability would have been caught in a formal proof, it also would have been caught by a code review. In fact, it was caught by a code review. Least Authority caught this vulnerability 2 years ago when they audited Ethereum, and there were several papers and blog posts written about it in the weeks before the hack. I think that mitigation strategies will take the form of defense in depth, with contract developers learning best practices and using frameworks to automate sensitive procedures, much like most databases come with the ability to do transactions, and avoid SQL injection.

Maybe there is a non Turing complete language out there that strikes the best balance between decidability and expressiveness. When it emerges, it will be possible to implement it on top of Turing-complete languages such as Solidity or EVM bytecode. It is not possible to go the other way. Like getting a haircut, it's a lot easier to take expressiveness away than to add it back. If language research progresses to such a point that it becomes easy to formally verify any program, while retaining the productivity of today's languages, it will greatly benefit all of computing. Until then, in smart contracts as in other areas, people will use code that runs and does what it needs to do with some reasonable assurance of safety.

Lightning

Another topic that Tony touches on is the Bitcoin Lightning network. The Lightning network is a network of payment channels using hashlock reveals to trustlessly move payments across intermediary nodes without posting any transactions to the blockchain. Tony feels that the Lightning network is overly complex and possibly insecure. I agree with this assessment, but not in the way Tony thinks. I actually think that the Lightning network is complicated because of the non Turing complete language that it is programmed in- Bitcoin script. Allow me to explain.

Once you get past the complications introduced by Bitcoin script, the concept of channels and hashlocks is simple. A channel is an escrow arrangement that allows two participants to exchange payments directly between each other without having to contact a bank or blockchain to process and clear the payment. Let's say that Alice and Bob want to open a channel. They go to the bank and open a special escrow account and each put in $100. They tell the bank

"Give this money back to us when you receive a note signed by both of us, with updated balances"

If Alice now wants to pay Bob $5, she writes a note that says

"Alice: $95, Bob: $105"

She signs this and sends it to Bob. Now Bob can consider himself paid. If he wants to get his money out of the channel, he signs the note as well, and gives it to the bank. The bank pays him $105 and Alice $95, and Bob is $5 ahead.

There's a problem: Someone can cheat. Let's say that Alice and Bob have been exchanging these notes back and forth for a while, and the last note read

"Alice: $195, Bob: $5".

If Bob wants to cheat, he could just take the first note to the bank and receive $105. To combat this, we use a sequence number and a "challenge period". When Alice or Bob take a note to the bank, the challenge period starts. If during this challenge period, a signed note with a higher sequence number is submitted, the bank honors this note instead. This way, as long as Alice and Bob check in with the bank at least once every hold period, they can stop each other from cheating.

This mechanism lets Alice and Bob exchange as many payments as they want, as fast as they can send notes back and forth, within the amount that they have placed in escrow, without having to trust each other. But what if Alice wants to send a payment to someone she doesn't have a channel open with? Let's say that she wants to send a payment to Charlie, and Charlie has a channel open with Bob. She could send Bob a payment, and trust him to send Charlie a payment of the same amount. What if Alice wants to send Charlie a payment without having to trust Bob? This is possible with something called a hashlock.

Alice sends Bob a note with balances updated to give Bob a payment which says

"Only honor this note if Bob presents the preimage to this hash: 2bb80d537b1da3e38bd30361aa855686bde0eacd7162fef6a25fe97bf527a25b"

She sends the preimage to Charlie. Now Bob can only get his money if he can find the preimage. To get Charlie to give him the preimage, he prepares a note like the one that Alice sent him, locked with the same hash. For Charlie to unlock this payment, he must reveal the preimage to either the bank (if he's going to close the channel at this step) or to Bob. Once Bob's got the preimage, he can do the same to get his payment from Alice. Payment setup goes forward, payment authorization goes backward. At each step, incentives are aligned so that participants do not need to trust each other. This kind of payment can go over an arbitrary number of intermediary nodes, and if there is a large enough network, one can theoretically get a payment to anyone on the network.

This is how the Lightning network is supposed to operate. Of course, in the case of Lightning, the actual procedures are much more complex. This is not because the creators of Lightning were being intentionally obtuse. It is because Lightning needs to work with Bitcoin script, a non Turing complete language. This same type of network is far simpler to implement on Ethereum. Turing complete languages allow humans to express a series of steps in a way that makes sense to us. Non Turing complete languages force the programmer to jump through a lot of hoops, and work around the language. Tony's complaints about the complexity of Lightning actually disprove his take on Turing complete languages from the previous section!

Tony asks "what if one of the hubs on the Lightning network gets popped?" Absolutely nothing happens to anyone using the hub for payments, although the operator of the hub could lose a lot of money. The entire system is premised upon the being able to move payments across nodes that one does not trust.

In an incentivized mesh network, some nodes will be stationary, and some will move. We can set up payment channels between the stationary nodes, and they can pay one another with those. But what about mobile nodes who may only be within range of one another for short amounts of time? A payment channel takes at least a few minutes to set up, if we're going to wait for a sufficient number of confirmations (or even longer if it's a conventional currency and we're dealing with a bank). This is why we have multihop payments. By routing a payment across several existing payment channels, we don't have to wait for a new channel to be set up.

Multihop payments can result in a lot of network traffic if we don't know what route the payment will take. Depending on the payment routing protocol, there may be a search of the network for every multihop payment (RPR, a payment channel routing protocol, currently does this).

Instead of having smart conditions return either true or false we can have them return a fraction (see Flying Fox). The transfer amount of the hashlock is multiplied by the fraction. This way, smart conditions have fine grained control over what fraction of a payment gets transfered. This gives us the option to build more sophisticated smart conditions.

The proportional hashlock gives us the ability to build a sort of higher level micropayment channel that sits on top of a multihop payment which is released incrementally. Its properties are similar to a low level payment channel in that individual payments are very cheap, and either participant can leave at any time and get all the money they are entitled to. However, this channel only goes one way.

Proportional hashlock smart condition:

condition(secrets)
  // An array of 1000 hashed secrets
  hashes = [xyz123, abc789, ... ]
  numerator = 0

  // A 'for' loop that will run 1000 times
  for i in 0..1000:
    // If the hashed secret is correct
    if sha3(secrets[i]) === hashes[i]:
      // Add 1 to the numerator
      numerator++

  // Return the fraction of secrets that were correct
  // (the total transfer amount will be proportional to this)
  return numerator / 1000

(roughly translates as)

List of 1000 hashes: xyz123, abc789, ...

Numerator: 0

Take this list of hashes and compare them to a list of secrets that you
will be given. Hash each secret in the secrets list, and compare it to
the corresponding hash from the hash list. If they match, add 1 to
Numerator.

return Numerator / 1000

With a proportional hashlock, the amount of payment released is proportional to the number of correct secrets that are supplied when the condition is evaluated. Here's how to use it to make an incremental multihop payment:

Alice -- Bob -- Charlie -- Doris -- Erin

Alice would like to pay Erin.

• A large multihop payment is set up from Alice to Erin, using payments with proportional hashlocks.
• Alice sends Erin secrets proportional to the amount that she wishes to pay. For instance, with the example smart condition above, a single $1000 payment could be released in $1 increments.
• Each time Alice wants to pay a dollar, she sends the Erin one of the secrets. Erin checks whether it works to unlock a fraction of the payment.
• If it does, it considers the payment complete (At this point, the channel could be closed and Erin would be able to redeem 1 1/1000 of the channel amount, or $1).

Caching

What if Erin caches secrets without passing them on to Doris? If Doris then sends Erin another payment on that channel that would result in Doris not having enough liquidity to honor the cached secrets, Erin is incentivized to refuse that payment (and possibly request that Doris close out the channel and open a new channel with more liquidity). Until that point, Erin can cache the secrets as much as she wants. This is good if the number of individual payments is very high.

Closing the proportional hashlock

What happens if Alice and Erin never make another payment? Doris can request that Erin sign an update transaction eliminating the proportional hashlock.

What if Charlie wants to close the proportional hashlock? He can let Doris know that he wants to do this. At this point Doris can request that Erin sign an update transaction eliminating the proportional hashlock. If Doris does not do this, then Charlie can simply close the channel between him and Doris.

Possible exploit? Alice sends the secrets to Charlie:

What if Alice sends secrets to Charlie instead of Erin? Charlie will be able to redeem the proportion of the payment that Alice sent him the secrets for, and so will Bob. In effect, Alice is paying Charlie. Now, if Alice sends those same secrets to Erin, Erin can use them to get a payment from Doris. Doris can use them to get a payment from Charlie.

So Charlie can't trust a payment from Alice, unless he can get Doris to agree to close their proportional hashlock immediately afterwards. But this doesn't matter, because Charlie is not supposed to be receiving the secrets from Alice anyway.

What is it good for?

The actions taken in this scenario, and the incentives, are similar to a scenario where Alice sends Erin a series of separate normal multihop payments over the same set of intermediary nodes. The difference is that a payment of a certain size can be set up beforehand and then released slowly, with secrets flowing in one direction, without new payments needing to be set up. The secrets are also cache-able, and can be released in blocks. Also, the payment can be routed as one payment, which can reduce routing traffic. A lot of these goals could probably be accomplished in the routing protocol, but depending on how it was done, they could compromise the strong anonymity guarantees offered by routing protocols such as RPR.

For instance, incentivized mesh networks. These are networks of independently owned routers which are automatically compensated by the network. In an incentivized mesh network, individuals could invest in providing internet service to their neighbors without the overhead of an ISP business. Also, due to the fact that individual routers compete with one another in such networks, consumers would be free from the monopolistic pricing which characterizes existing ISPs.

Incentivized mesh networks function by having routers pay their neighbors to forward packets. Nodes could pay per packet, but this would normally impose a huge overhead. Processing these payments with conventional currencies would result in an at least an API call to the payment processor with every payment. It would also likely result in a database entry at the payment processor. Processing these payments with a cryptocurrency (saving each and every one to the blockchain) would quickly result in a very bloated blockchain. Payment packets would vastly outnumber data packets.

Nodes could keep a tally of payments to one another to reduce the total number of payments. This requires nodes to trust one another. If I am paying you to forward my packets, either I put a deposit down with you, or you extend me credit. In the former scenario, I need to trust you, while in the latter, you need to trust me. The less the amount of credit or deposit is, the larger the number payments will need to be sent.

Payment Channels

What if there was a way to transfer money without trusting the other party, and without incurring the costs of a blockchain or conventional payment? Payment channels can do this. In a payment channel, two parties deposit money with a third entity that both trust. If the channel is to transfer conventional currency, a bank or payment processor plays the role of trusted third party and holds onto the money. Both parties must trust the integrity of that bank or payment processor. If the channel holds cryptocurrency, a contract on a blockchain locks the funds from both parties. Both parties must then trust the integrity of that blockchain.

The bank or the blockchain will transfer the locked funds back to the channel participants upon receiving a message signed by both.

Upon receiving this message, the bank or blockchain also updates the amounts to be transfered back. If Alice and Bob both deposited $100 to open the channel, and close it with balances of $95 and $105, Alice has effectively given Bob $5. So, to pay Bob, Alice signs a message updating her balance to $95 and Bob's balance to $105. She sends this message only to Bob, without contacting the bank or the blockchain that the channel is open with. If Bob wants to get his money out, he simply posts the last signed message to the bank or the blockchain.

There's one issue though- someone could cheat. Let's say that Alice makes a payment to Bob and the balances are updated to Alice- $50 and Bob- $150. Then Alice makes a payment to Bob, reversing the balances to Alice- $150, Bob- $50. Bob could take the old message where he has $150 and post it, cheating Alice out of $100.

How to prevent this? We need some way for the bank or the blockchain to find out whether a message represents the account balances that Alice and Bob most recently agreed on. If Alice and Bob put a sequence number on each message and increment it every message, either of them can prove if one message is more recent than another. If the bank or blockchain then waits a certain length of time (or "hold period") before transferring the money back, it gives either party a chance to prove that the other is cheating.

What if Alice and Bob don't want to wait to get their money out? They can simply sign a message with a hold time of 0. This means that the bank or blockchain will immediately transfer the money to their accounts. The only situation in which the hold time will actually be a factor is a situation where one of the parties wants to close the channel and the other is unresponsive or uncommunicative.

This is the foundation of Universal Payment Channels (technical details in the white paper).

Multihop payments {#multihop}

Back to the incentivized mesh example: If Alice and Bob both have nodes in an incentivized mesh network, they can open a channel in whatever currency they wish, and exchange packets and payments to their heart's content. But what about mobile nodes? Alice has a cell phone, and happens to walk into range of Charlie's wifi hotspot. Charlie could forward Alice's packets on to their destination, but he's going to need some payment from Alice. Charlie and Alice don't have a channel open- how will Alice pay Charlie?

What if both Alice and Charlie have channels open with Bob? Alice could send Bob a payment, who would then send Charlie a payment. But now Alice needs to trust Bob.

Smart conditions and hashlocks

We have to make sure that Bob can't steal the money. UPC allows us to make payments with pieces of code called "smart conditions". The bank or blockchain evaluates the smart condition, to find out whether it should transfer some money. We can make a type of smart condition called a hashlock which allows us to trustlessly route payments through one or more intermediary nodes. A hashlock basically says: "transfer this amount of money if you are given the string that hashes to this hash".

A hashing function turns whatever value you give it into a random-looking string of characters, known as a "hash". For instance, the string "alfred" might be turned into "d8si32" by a hashing function. This has two interesting properties: the same hashing function will always turn "alfred" into "d8si32", and someone who has "d8si32" has no way of knowing that it was derived from "alfred". So, if Jim wants Gregory to be able to recognize that someone has the correct secret word, but Jim doesn't want to actually give Gregory the secret, Jim can give Gregory a hash of the secret. If someone gives Gregory a word they claim is the secret, Gregory can hash it and find out whether it is the correct word.

Trustless multihop payments with hashlocks

To route a payment through Bob, Alice sends Charlie a secret, and the amount of the payment. Alice then sends a payment to Bob, hashlocked with the secret she sent Charlie. Bob sends his own payment message to Charlie, hashlocked with the same secret, and containing a payment of the same amount.

To unlock the payment from Bob, Charlie reveals the secret to him, which allows Bob to unlock the payment from Alice.

This all happens as fast as packets can be forwarded, and doesn't store anything on the blockchain or bank servers. It's an ideal mechanism to handle the high volume of payments that need to be processed by an incentivized mesh network.

You may be wondering- how do we know that Bob is the best intermediary node to route payments from Alice to Charlie? Stay tuned for the next post, about Reactive Payment Routing, a routing protocol which finds the cheapest path for payments.

For a more complete overview of UPC, see the white paper.

Acknowledgement: Payment channels utilizing a hold period in this way were, to the best of my knowledge, first introduced in Zackary Hess's Flying Fox.