The big issue in consumer broadband is Internet access vs. Interweb access. Internet access is what businesses get when they order bandwidth over a traditional leased line---T1, OC12, and so on. Optimistically, professors also get Internet access through the Ethernet jack in their office's wall, although recently I think this is less true as campus IT departments become more politically powerful, blindly security-obsessed, and business-oriented rather than academic-oriented.
Interweb access is what ``consumers'' get shafted with in their homes as so-called broadband. Most corporations wire their employees' desks with Interweb access, even though the corporation itself gets an Internet connection from its ISP.
Here are some problems that distinguish Interweb access from Internet access:
Internet access by definition never logs the content of communication. Whenever such logs are created, an Internet carrier considers this a penetration of the network's security. Internet carriers use ``sniffers,'' portable and stateless standalone devices that log communication content, but the logs are destroyed when the sniffing session ends, which will happen at least as often as the sniffer's operator gets tired and has to go home for sleep. Internet carriers routinely log the quantity of communication with packet counters or byte counters. These counters can distinguish protocols---ex., web access from email access---and they can distinguish one customer from the other. However, hopefully they do not distinguish both things at the same time. Particularly fancy counters can count lost-packet retransmissions or round-trip ``ping time'' variance.
An important characteristic of Internet logging is that it's real-time. Internet carriers plug in the sniffer or set up packet counters after they have a question they want to answer. Interweb carriers try to log as much information as practical in case they think of a question later. In the police business, this is the difference between an investigation and a fishing expedition. This is an interesting analogy, although of course probably only a few of the Internet carrier's investigable questions will involve crime.
Cable Interweb carriers in particular have a reputation for keeping detailed logs of their customers' communications (what web sites they visit, what links they follow) which they claim is necessary to ``characterize'' the traffic on their network---in other words, to manage their network, they want to log the content of your communications. This makes me wonder, what type of management decisions require developing this type of database? Well, duh: highly-invasive decisions that discriminate in the quality, the type, or the ``privilege'' of network access based on the content of your communications. We'll see some of these management decisions later.
One of the big problems with the Interweb logging style is not retribution from or privacy invasion by the carrier itself, but rather the liability that the mere existence of their detailed dossier creates for their customers. With the USA Patriot act and the ongoing repurposing of the US Executive Branch for crime prevention (Foucault's ``control society'') rather than crime investigation (Foucault's ``punishment society''), the communications log database is available to most government agencies without a court order. This is why Indymedia tries to minimize the logs they keep to deal with repeated government demands for lists of visitors (2) (3).
The EU has likewise changed its formerly famous privacy laws. Conventionally, citizens are somewhat dependent on carriers to protect fourth-amendment rights because anyone can ``cooperate'' with a government investigation even if they are not legally compelled to provide surveillance copies of their databases. The carrier might do this because it reduces their legal bills, or they might sell their logs of your communications to the government like a credit reporting agency. However, European carriers used to be forbidden from retaining customer data like call logs and web access logs longer than necessary for billing, meaning their ability to ``cooperate'' was legally limited. Now, this provision is completely reversed: European carriers are required to retain this information for one or two years, and they're also compelled to share the information with government investigative and tax agencies upon request, without the judicial review that compulsion used to require.
If you are completely apolitical---not even curious about politics in which others are involved---and you don't happen to fit any ``suspicious'' electronic profiles, and you are uninterested in asserting your legal right to share music with family members under the AHRA, and you don't mind losing your fair use rights under the Copyright Act, and you have no qualms about unilaterally imposing your submissiveness upon everyone with whom you share the Internet connection you're buying, then these privacy issues may not interest you personally. However, as a moral person you should ask yourself in what type of world do you want to live, and what type of world do you want to create for your neighbors and children?
If this seems political, it is. The Internet is a political place, and these politics have pervasive effects on our daily lives.
Some inexpensive Internet connections have contracts that forbid their recipients from reselling them---for example, a University professor can't stick a web server in his office and start his own business selling web pages. However, the Internet customer is free to share bandwidth with anyone in his or her community: a University grows its campus network as it builds new buildings without renegotiating with its ISP, and a business is free to set up an employee modem pool to give employees company 'net access from home, or purchase a leased line between the New York office and the Akron, Ohio office and extend their Internet connection to that link. The customer gets to define his or her community, not the ISP.
Interweb connections usually have heinously onerous contracts that forbid ``redistribution'' rather than resale. The carrier claims more than just the right to exclude their customer from a specific retail business in competition with the carrier. Rather, to buy Interweb access the customer must agree that the carrier has the right to dictate the topology of the customer's network and who is in the customer's community. They might make requirements about the physical size of your network---one ``residence.'' They might say that your community is one ``household,'' as defined by income tax filing I suppose, so you can't legally share the connection with anyone unless they are also dependent upon you for food and shelter. Some Interweb contracts tell you how many machines you're allowed to connect, or they have a ``no hubs'' policy.
It's not an exaggeration to look at contracts prohibiting redistribution this way. Both cable and telephone companies once had marginal charges for more than one TV set or extension. Recently this contract scam has become a hot issue because of (1)(2)wireless community networks, which exist in basically all major US cities. Some DSL carriers explicitly allow these communities, while other DSL carriers and (AFAIK) all cable carriers do not.
I think it is partly a business issue. DSL probably has significant marginal cost per subscriber, while cable carriers may spend mostly on coverage, with minimal marginal subscriber cost until the network gets crowded. It makes sense that cable would perceive these ad-hoc picocellular systems as a threat more so than DSL. When the DSL ISP that writes the contract and the telco that sells the physical loop are separate entities, the incentive to sell gratuitous wired connections could be even less: for example, one ISP mentioned in the wireless community survey above charges $5 extra for busy wireless nodes.
Anyway, that's all speculation, and none of it is the user's problem anyway. Users should not be contractually obligated to support a particular business model. It should be the other way around.
Anything that doesn't let all these packets through, and also other kinds of IP packet that haven't been invented yet, isn't Internet access.
Interweb access almost universally forbids at least everything except TCP, UDP, and ICMP, which squashes much of the Internet's experimental potential. This also prevents some popular applications---VPNs won't work unless the Interweb carrier adds IPsec to the blessed list. The IPsec community is working on a hack to encapsulate IPsec in UDP so that it might slip past some of these Interweb filters, but the hack doesn't work very well---it breaks Path MTU discovery, and probably has some other problems, too. The UDP NAT-Traversing hack (problems) (draft) in IPsec is one example among many of half-broken twisted protocols that exist for no other reason than to evade arbitrary Interweb restrictions.
Interweb access is usually much worse than just blocking non-TCP/UDP/ICMP packets, in that it isn't really guaranteed to let anything through except web pages. It might, for example, block all ``incoming'' connections: every TCP circuit has one connecter and one listener, and with incoming-connection-blocking Interweb machines can't be the listener, only the connecter. This means that any pair of machines on such an Interweb connection can't reach each other. PERIOD. They can only communicate through an intermediary, like a web server with forms and CGI pages. This makes any peer-to-peer system impossible for these Interweb machines, including file sharing, Internet telephones, PDA synchronization, and multiplayer games. But of course web browsing works fine.
Incoming-connection-blocking is the ultimate in port-blocking, and this is how I've seen Qwest and Verizon DSL operating. The Cisco 675 router/modem consumes the account's only legitimate global IP address. The Interweb customer can wire an arbitrary number of web-browsing machines to the router/modem with Ethernet. The router/modem assigns these Interweb machines fake IP addresses on the unroutable local-use-only subnets 192.168.0.0/16, 172.16.0.0/12, or 10.0.0.0/8, and it uses NAT to make these machines' web browsers appear to Internet machines as if they were running on the router itself. Since your friends' machines on another Qwest/Verizon Interweb connection will also have fake addresses, it's impossible for you to reach them without using an Internet machine as a mediator.
Finer-grained Interweb port blocking, like that which cable companies seem to favour, is usually designed to sabatoge specific disfavoured types of communication rather than all incoming connections. For example, almost universally they will make sure you can't run your own web server or your own mail server. Why? Who knows? Maybe they are just sadistic bastards. But the effect is to silence the customer---he or she can ``consume content,'' but cannot publish.
There are new protocols designed explicitly for working around Interweb port blocking and for no other reason, like httptunnel, NAT-T IPsec-inside-UDP, and MS-SOAP(?). These aren't very liberating because you have to prearrange communication partners that are ready to accomodate you with these cumbersome and bandwidth/latency-wasting protocols. Also, the protocols still won't work at all between two Interweb machines---they only work between an Internet and an Interweb machine.
Finally, the protocols usually don't work very well---for example, NAT-T IPsec-inside-UDP breaks the Internet's Path MTU Discovery protocol, which causes all kinds of weird problems like TCP circuits that transmit the first application request (ex. http 'GET /' or ftp's command circuit) but then get stuck once the bulk data transfer starts (ex. the http response or the ftp data circuit); and httptunnel has problems where one end closes the tunnel because of some Interweb timeout, but the other end doesn't realize the tunnel is closed so you have to wait several minutes for a timeout before manually reopening the httptunnel---then of course there is the obvious incredible slowness and loss of the TCP_NDELAY option when you do TCP-inside-TCP. The Interweb workaround protocols represent a ridiculous, terrible, and technically-unjustifiable situation.
Some corporate IT departments retain the block-and-proxy style of firewall, usually because they want to block web sites by URL or or make logs on employees' web access. Packet filter firewalls don't have that granularity---at best, they can permit or prevent access to the entire web or an entire web server, not individual pages. Also, their per-packet logs are too voluminous yet not detailed enough. Packet filter firewalls are most effective for controlling the people outside the firewall, while corporations are increasingly interested in controlling people on the inside, a task for which only the older block-and-proxy firewalls are suited.
Unfortunately, block-and-proxy firewalls have some limitations:
The transparent proxy ``solves'' all these problems. Like other proxies, it's installed on a router between the proxee and the Internet. But unlike other proxies, transparent proxies don't block packets that aren't directed to the proxy. Instead, the transparent proxy captures these packets and redirects them toward the proxy. This immediately solves the annoyance problem---no need to configure the proxy's address into the web server. It also solves the obvious-to-the-user problem, since a cleverly-designed proxy that correctly mimics web sites that are down is virtually impossible to detect, and most users aren't clever enough to notice even a proxy that doesn't mimic unreachable web servers properly.
The transparent proxy also addresses the performance problem by cooperating with a firewall. Interweb carriers can, for example, have one firewall but several transparent proxies---the firewall feeds traffic into the proxies, balancing it across them. The proxies intercept the first few packets of the HTTP request, enough to log the page load and possibly block it, then instruct the firewall to pass the rest of the HTTP request's packets unmangled, without bothering the proxy. This way, most packets get handled by the low-overhead packet-filtering firewall, but the first few packets in the session needed to implement URL blocking or detailed surveillance still go through the proxy. Not all transparent proxies work this way, but those that do can potentially handle large traffic loads with less computing machinery than the old block-and-proxy firewalls.
Transparent proxies do a great job of restricting the content of Interweb users' communications even more than a packet-filtering firewall can, and also of generating the detailed and increasingly-invasive logs that Interweb carriers want. And they do it covertly.
In addition, transparent proxies can implement the more nefarious plans described in the nettime posts below, like sabatoging adaptive-bandwidth streaming sessions to shrink the bandwidth upon which they dynamically settle(1), or providing preferential service to web sites allied with the Interweb carrier's parent transnational corporation(2). Transparent proxies are highly invasive, meaning that they mediate all of an Interweb user's communication with powerful and complicated software to an extent that even packet sniffing cannot accomplish, and they allow surveillance and control on an unprecedented granularity. Packet sniffing alone is a serious violation for an Internet user, but this is much worse.
Implementors of transparent proxies sometimes justify them by claiming they contain a ``cache'' which reduces load on parts of the core network when several customers retrieve the same static web page. Yet, I've never seen a transparent proxy user offer any statistics to back up this claim, and the efficacy of cacheing is limited by load-balancing, and finally and most damningly, if there is really a speed improvement then why does the proxy need to be transparent? Another justification implementors offer is that some transparent proxies block some Microsoft Windows viruses---again, why does the proxy need to be transparent? Please note that it's impossible to realize the performance benefits over block-and-proxy if the transparent proxy must do cacheing or virus-filtering. There really is no justification for a covert proxy that claims to exist only for its users' benefit.
I first read a detailed review of these evil devices in the 2002-07 issue of PC Magazine, but of course they are at least a couple years older than that, and most of the devices PC Magazine reviewed are small-scale things while a real Interweb carrier would probably not use one of the reviewed low-end devices but rather a set of devices that balance the load across themselves and integrate with a Checkpoint firewall, while centralizing the logging database of course.
| xDSL | upstream | downstream | total | IPs | price |
|---|---|---|---|---|---|
| ADSL | 128kbit/s | 1500kbit/s | 1628kbit/s | 4 | $76/mo |
| sDSL | 384kbit/s | 384kbit/s | 768kbit/s | 4 | $120/mo |
sDSL is almost twice the cost, even though it has less than half the total bandwidth. Cable is also bad in this regard, since you basically pay only for upstream bandwidth, although they may throw some nonsense into the package to cloud this issue such as promises that the Gestapo will be more lenient in interpreting the meaning of ``excessive usage'' if you pay for a higher upstream cap.
First of all, a carrier should not regulate the content of its customers' communication. This is called censorship. It's not subtle or complicated!
Second, the resulting service cannot be sold as Internet access, since they've forbidden use of the bulk of common Internet applications, including half of the two most popular applications, web and email (``no servers'').
Third, the Internet does not recognize the distinctions they make. All Internet applications are ``peer-to-peer'' applications because all machines connected to the Internet are peers. The concepts of ``servers'' and ``peers'' exist only because of the boundaries created by Interweb accounts: the Internet is no longer populated by peers---rather, some machines are Internet and others are Interweb. For example, a common technical restriction discussed above is that Interweb machines often may not have any unmediated conversations among themselves---all communication must terminate at or at least pass through an Internet machine. The legal restriction created by this policy is not even linguistically parsable without the Interweb these carriers have created through their other technical restrictions. The policy is, in effect, ``you may not circumvent the effect or the spirit of our other censorship mechanisms.''
Likewise, Internet users are traditionally free to develop their own software. Therefore, there is no network-wide distinction between a ``bot'' acting on a user's behalf and an ``application'' acting on a user's behalf. The distinction can arise only in specific communities like IRC where it refers intimately to the content of an IRC conversation---whether English text is typed by a human or is a canned, pre-recorded message like an answering machine tape. This distinction is something that only human beings on an IRC network can make: many IRC clients include lots of text-generating automated feechurs that respond to a human's keyboard which are not considered ``bots.'' A community invented the term ``bot,'' and the Interweb carriers co-opted it and decontextualized it, taking full advantage of its vagueness.
What next, ``no answering machines''?
I think what comes next after no ``servers'' is, ``no writing or installing your own software---you may not run or install any software which has not been signed by The Carrier's cryptographic key. If you use such software, you may not be connected to the carrier's network, and you may not use any information as input that was obtained while connected to the carrier's network.'' Hardware architectures like MS-TCPA can enforce such a contract provision, and will no doubt claim they're doing so for your own good, to protect you from the many ``dangerous'' Microsoft Outlook and Microsoft Internet Explorer viruses on the Internet. (I think it would be more effective to quit using Outlook and Explorer!)
But broadband connections are ``always on.'' Even some of the TeeVee advertisements for Interweb access mention this feature. Interweb broadband usually uses DHCP or PPPoE, but because the connection is always-on the ``dynamic'' IP address is never unassigned. There is no IP address savings over static addresses.
Nor is there any ``network management'' advantage, because DHCP and PPPoE could just as easily assign consistent static addresses as dynamic ones.
It is a ploy, designed to make it impossible for others to reach your machine without some intermediary ``server'' to keep track of your musically-changing address, to prevent you from running a DNS server, and to interrupt any long sessions like telnet or ``instant messaging.'' My address has changed thrice in six hours on the Verizon DSL Interweb account that I'm leeching through someone's 802.11b to write this page, thrice kicking off my ssh session even though the connection has been ``always-on'' the whole time. But of course web browsing works fine.
Verio has a pattern of disconnecting customers upon receipt of the first demand letter from any lawfirm, without informing the customer or providing an opportunity to beg, much less other reasonable speech protections like:
Interweb carriers follow the Verio model---just as they ``cooperate'' with police, during government investigations and fising expeditions, they comply unilaterally with civil threats, baseless or sound. Due process is reserved for subscribers with big enough legal teams or large enough monthly bills to have bargaining power. Spammers like MonsterHut get injunctions so their connection can't be shut off, while individuals get thwacked without so much as a phone call.
This is partly the fault of the legal environment. For example, if carriers were legally obligated to give their subscribers certain considerations when someone complains about the content of their subscribers' communications, then the carriers would also have less liability for acting to meet these obligations. XS4ALL mentions the European Commerce Directive, which may be such a law, in their case with Deutsche Bahn. Yet I think the Deutsche Bahn case still must have been expensive for them.
However, even in today's legal environment, some connections are already Internet connections and give the subscriber the due consideration, while others are Interweb---you can't ``get in trouble'' for sheeplike web browsing, but any other activity is fair game to shut down your connection---preemtive censorship---without even bothering with the formality of pounding you in the ass with cripplingly expensive civil lawsuits.
Hughes DirecWay discovered 5% of the users are using 50% of the bandwidth, so they automatically throttle the bandwidth of this 5% down to modem speeds (``FAP'' them) long enough to bring them under their 95th percentile average bandwidth. ``Excessive'' in this case is defined by your neighbors. ``Fairness'' means that those with different use patterns than most users must be punished and forcefully brought into conformity. Hughes goes so far as to refer to DirecWay's ``intended purpose'': web browsing. This is rather a different definition of ``fair'' than the QoS literature, which interprets fairness as giving every user equal access to bandwidth, averaged on as short a time-scale as possible and independent of what protocol they use or how many connections they open at once---a very different metric than Hughes's 1 - 4 hour average and fluctuating cutoff.
But DirecWay is actually an unusually-favorable ``excessive'' use policy, because it's quantitative (although the actual quantities may fluctuate as the months pass, and aren't exhaustively disclosed), and because it's partly implemented by a machine that will apply the (admittedly, biased towards web users with bursty access patterns) policy evenly to all subscribers. Most Interweb accounts forbid ``excessive use'' without quantitatively describing excess. Interweb carriers probably hope that, when we compare AUPs, we will interpret these ``excessive'' provisions as similar in meaning to Hughes's FAP: normal 95th-percentile web browsing is not excessive, while any more intensive use pattern is excessive. That way, 95% or more of their potential subscribers will think to themselves, ``I have nothing to fear, because I am no 5% freak. I just want to `get on line.' '' In practice, the ``excessive use'' contract provision operates as a ``don't get noticed'' clause, leaving the carrier free to threaten or kick off anyone who comes to their attention through mining their content-logging systems.
It's illogical that Interweb carriers would apply the ``excessive use'' clause evenly across subsribers or time. They have a free hand to kick off the heaviest users whenever the economy gets tight or their core pipes become oversubscribed, but keep these users when bandwidth is cheap and plentiful. Likewise, AOL/Time-warner/20th-Century-Fox Cable is free to claim that your FTP or Hotline traffic is ``excessive,'' but that the same amount of HTTP traffic is not ``excessive'' for other customers. You don't have enough information to prove their real motive for kicking you off, and even if their motive were suspicious it still might be of no legal help to you given the vagueness of ``excess'' and the lack of any requirement that they treat subscribers equally.
A ridiculously optimistic ``Ned Flanders'' sort of user might not only accept the ``excessive use'' doctrine, but extend it into appologetics for the other Interweb restrictions. For example, he might claim that usurous upstream prices, server port blocking, musical IP addresses, and speech-limiting contracts exist because they must have somehow determined that servers tend to use more bandwidth than sheep. I'd call this affirming the consequent, but let's accept it temporarily so that we can undermine it with another argument. Likewise, let's accept redistribution prohibitions. They might be about trying to cram your connection into an invariant usage profile. Although they don't force you to disclose how many kids you have or whether you fit into the sex, age, and marital status of a pornography glutton before writing the policy, let's temporarily accept ``excessive use'' as an apology for redistribution prohibitions as well.
If Flanders is right, then there's a simple solution to this Interweb mess: start charging your users per-gigabyte. If this happened, we would have price competition for core bandwidth. Once users were able to openly compare two services by price rather than by tricky contract legalese, the per-gigabyte price would plummet and ``excessive use'' as a justification for the Interweb would crumble. Just look at the marginal bandwidth prices on those links that do charge per-gigabyte, like web hosting. They're a joke! In case you haven't noticed, the planet has a laughable oversupply of inexpensive core bandwidth.
I'd like to see services that charge per-gigabyte. I'd like the accounting to be open, like electrical power meters: mount a display on the DSL router or the cable bridge. The display should read in currency, and maybe it should synchronize with the central accounting system like the timers on some CDMA celfones so that it matches the customer's bill exactly. Customers will not fear the meter when the dollars and cents are right under their noses. And I'd like carriers to get rid of all this other garbage. Once you offer to pay for your bandwidth, the legalese instantly becomes obvious censorship rather than some mysterious unchallengable fascist ``policy.''
IMHO, the Interweb issue is the big, killer, show-stopping problem with cable Internet. and I will never touch cable so long as this problem stands. I would sooner spend my $55/mo on a CDPD account, which comes with an always-on unfiltered static IP address (albeit at a crawling 1 - 10 kbit/s) than on cable. At least the address is static and unfiltered, but Verizon CDPD still has ``excessive use'' provisions and a ``no servers'' policy!
Of course, not every Interweb account adopts every one of these oppressive tactics to prevent your full participation in the Internet. Some accounts are worse than others. That's why, right now, DSL is better than cable. With DSL, you can choose either your local phone company or Covad to provide the physical ``Digital Subscriber Loop'' and the ``modems'' on each end of it. Then, no matter whether you choose Covad or the local phone company, you can make a second choice among many ISPs that can route your traffic onto the Internet. Covad vs. the local phone company determines most of the ``different prices for upstream vs. downstream bandwidth'' scam (both are guilty of it), while your chosen ISP will implement all the other Interweb tricks.
Some cable Internet lets you choose among ISPs and some doesn't. The situation isn't stable---sometimes the cable company eliminates their ``competitors'' through agressive contract pricing, then takes over the ISP job for customers of those ruined competitors (<cough>excite@home<cough>). And my impression is that competing cable ISPs, where they exist, differ much less w.r.t. the Interweb issue than DSL ISPs.
Right now, I think your best choice is Covad DSL with a well-chosen local ISP like BWay or Pilosoft in NYC that's negotiable or at least responsive to small groups of non-Interweb customers. There is also some argument for a national ISP's better bargaining position with the ILEC or Covad, if they have some positive reputation like Speakeasy, but most don't.
Of course there is a seperate, largely irrelevant dialog about cable-vs.-DSL, but I think the extent of ISP competition and how it addresses the Interweb problem dwarfs all the other issues.
The bottom line is, users should not swallow these connectivity ``packages'' uncritically based on useless promises of ``download speed.'' They do so without any concept of why broadband is revolutionary. In some markets, broadband is the same price or cheaper than POTS modems, but in markets where it's more expensive users should at least want real justification of the increased price, not just faster loading of those web pages that use JavaScript to suppress the text until the banner ads appear.
Unfortunately, choosing an ISP is difficult. You can try yourself to read their AUP (which changes from a ``policy'' into a contract when you're forced to sign it before they will give you Interweb access). You might spot the onerous contract provisions or admissions of Interweb tactics yourself, but of course this is difficult. You can also take a technical perspective and look for favorable buzzwords like ``Static IP'' and avoid unfavorable ones like ``PPPoE --- PPP over Ethernet.'' But I think making a good ISP choice is not something users---myself included---can reasonably accomplish. The information is too hard to find since it's often not well-covered on the web pages, and the guys at the toll-free pre-sales number never, never have a clue about any of this stuff---their first question is usually ``Are you running Windows 98 or Windows 2000?'' followed by ``what version of Internet Explorer and what URL are you having problems with?''
The ISP choice is difficult to make, and not everyone even has this choice. Many DSL carriers, in the US and worldwide, adopt the same Interweb tactics as the cable companies. Check out this digest from the pinko commie list ``nettime'':