French Company Potentially Could Solve Balkanization of the Internet” 🇫🇷

Years ago now Google quietly announced its “Loon Balloon Project” in New Zealand. The objective was to launch high altitude balloons that could potentially float over areas of the globe that did not yet have Internet access. The tech press predicted that the idea was “loony” indeed, though some called it “crazy cool.” Google has since also dabbled with the idea of low earth orbit satellites to achieve the same goal. With the rise of SpaceX, this seems an even more interesting technological approach, though other firms in the 1990s lost large amounts of money and failed.  A modest aerospace company and a subsidiary of Airbus in Toulouse France is manufacturing low-orbit internet access satellites, hoping to launch as many as 650 such satellites. The idea that is captivating me is the potential for space-based Internet access to potentially provide an alternative to growing political and corporate control and Balkanization of the Internet.


Net Neutrality May Yet Be Achievable…Maybe

Years ago now Google quietly announced its “Loon Balloon Project” in New Zealand. The objective was to launch high altitude balloons that could potentially float over areas of the globe that did not yet have Internet access. The tech press predicted that the idea was “loony” indeed, though some called it “crazy cool.” Google has since also dabbled with the idea of low earth orbit satellites to achieve the same goal. With the rise of SpaceX, this seems an even more interesting technological approach, though other firms in the 1990s lost large amounts of money and failed.  A modest aerospace company and a subsidiary of Airbus in Toulouse France is manufacturing low-orbit internet access satellites, hoping to launch as many as 650 such satellites in a “global constellation”. The idea that is captivating me is the potential for space-based Internet access to potentially provide an alternative to growing political and corporate control and Balkanization of the Internet.

OneWeb Launches First Six Internet Access Satellites

Ariane Soyuz rocket launch with six OneWeb satellites on board. February 27, 2019

Political Internet Censorship And Access In Developing World Potentially Solvable

Aclear plastic box the size of a sofa sits in an underground factory in the suburbs of Toulouse in southern France. Inside it, a nozzle fixed to a robot arm carefully drips translucent gloop onto bits of circuitry. This is to help get rid of excess heat when the electronics start to operate. The slab that is created is then loaded onto a trolley and taken away as the next piece of electronics arrives for the same treatment.

This is what the mass production of satellites looks like. Making them in quantity is a necessity for OneWeb. The company was founded in 2012, and it has yet to launch a single satellite. Yet it plans to have 900 in orbit by 2027. That seems a tall order. Intelsat, the firm which currently operates more communications satellites than any other, has been around for 54 years and has launched just 94.

OneWeb, which is part-owned by Airbus, a European aerospace giant, and SoftBank, a Japanese tech investor, needs such a large quantity of satellites because it wants to provide cheap and easy internet connectivity everywhere in the world. Bringing access to the internet to places where it is scarce or non-existent could be a huge business. Around 470m households and 3.5bn people lack such access, reckons Northern Sky Research, a consultancy. OneWeb is one of a handful of firms that want to do so. They think the best way to widen connectivity is to break with the model of using big satellites in distant orbits and instead deploy lots of small ones that sit closer to the ground.

The rate at which an object orbits depends on how far away it is. At a distance of 380,000km, the Moon takes a month to travel around the Earth. The International Space Station, around 400km up, buzzes round in an hour and a half. In between, at an altitude of about 36,000km, there is a sweet spot where satellites make an orbit once a day. A satellite in this orbit is thus “geostationary”—it seems to sit still over a specific spot. Almost all today’s satellite communications traffic, both data and broadcasts, goes through such satellites.

The advantage of a geostationary orbit is that the antennae that send data to the satellite and those that receive data coming down from it do not need to move. The disadvantage is that sending a signal that far requires a hefty antenna and a lot of power. And even at the speed of light, the trip to geostationary orbit and back adds a half-second delay to signals. That does not matter for broadcasts, but it does a little for voice, where the delay can prove tiresome, and a lot for some sorts of data. Many online services work poorly or not at all over such a connection. And it always requires a dish that looks up at the sky.

Head in the clouds

Ships, planes and remote businesses rely for internet connections on signals sent from geostationary orbit, but this method is too pricey for widespread adoption. Beaming the internet via satellites orbiting closer to the planet has been tried before. The idea was popular at the height of the tech boom of the late 1990s. Three companies—Teledesic, Iridium and Globalstar—poured tens of billions of dollars into the low-Earth orbit (leo) satellite internet. It culminated in the collapse of Teledesic. Although the technology of the time worked, it was very costly and so the services on offer had to be hugely expensive, too. Iridium survived, but as a niche provider of satellite telephony, not a purveyor of cheap and fast internet access.

OneWeb is among several firms that are trying leo satellites again. SpaceX, a rocket company founded by Elon Musk, a tech entrepreneur, is guarded about its proposed system, Starlink, but on November 15th American regulators approved an application for 7,518 satellites at an altitude of 340km (bringing the total for which the firm has approval to nearly 12,000). Telesat, a Canadian firm, has plans for a 512-satellite constellation. LeoSat, a startup with Japanese and Latin American backers, aims to build a 108-satellite network aimed at providing super-fast connections to businesses. Iridium, still in the game, will launch the final ten satellites in its new constellation of 66 by the end of the year. Not to be outdone, a Chinese state-owned firm recently announced the construction of a 300-satellite constellation. In ten years’ time, if all goes to plan, these new firms will have put more satellites into orbit by themselves than the total launched to date (see chart).

These companies want to avoid the technical issues of geostationary satellites by putting theirs into a low orbit, where the data will take only a few milliseconds to travel to space and back. And because signals need not be sent so far the satellites can be smaller and cheaper. OneWeb claims they might weigh 150kg and cost a few hundred thousand dollars, compared with a tonne or more, and tens or even hundreds of millions of dollars, for the geostationary sort.

Floating in a most peculiar way

At 1,200km up, where OneWeb intends its first satellites to operate, they do not sit still in the sky. A satellite overhead will sink below the horizon seven minutes later. That has two consequences. First, to ensure that a satellite is always available to any user, a great many are required. Second, to talk to such a satellite you need an antenna that can track it across the sky.

One way to understand this is as a cellular-phone network turned inside out. On Earth, cell-phone towers are fixed; a user’s phone talks to the closest or least busy one, which may change as the user moves or traffic alters. In OneWeb’s system each satellite is a moving cell tower, circling the Earth from pole to pole in one of 18 orbital planes that look like lines of longitude (see diagram). The 900 cells, each one covering an area of a bit more than 1m kilometres, skim across the Earth at 26,000km an hour. Clever software hands transmission from one satellite to the next as they move into and out of range.

There are three ways to connect to such a network. One is to place an antenna on a terrestrial cell tower, which can use the satellites to get data to and from a mobile-phone network, in place of the fibre optic, microwave or cable links that are normally used. The second is for homes and businesses to have their own ground terminals, smaller and cheaper antenna that can talk to the satellite. The third is for vehicles to have ground terminals. This might be important for driverless cars, which will need to transmit and receive large volumes of data over an area which may be broader than that covered by appropriate terrestrial cellular networks.

In all cases data will make their way to the wider internet through large ground-based dishes, called gateways. An email sent from a house connected to one of the new satellite network, for example, would travel up to a passing satellite, down to a gateway then onward to its destination.

The firms involved today hope to overcome the obstacles confronting the previous generation of leo satellite firms because building and launching hundreds of satellites is now much cheaper. The cost of launch in particular has tumbled in the past decade with the arrival of better rockets and more competition. OneWeb has a contract, reportedly valued at over €1bn ($1.1bn), for 21 launches with Arianespace, a European consortium. Russian-built Soyuz craft will also take 34 to 36 satellites up at a time from either French Guiana or Kazakhstan. OneWeb may later use Blue Origin, a rocket firm owned by Amazon’s founder, Jeff Bezos; it also has a contract for launching single satellites to replace ones that break down with Virgin Orbit. Virgin Group, like Airbus and SoftBank, is an investor in the company. SpaceX intends to launch its satellites on its own rockets.

Space to grow

The bigger challenge is making satellites quickly and cheaply enough to fill up these rockets. It typically takes existing satellite-makers two years to build one after contracts are signed. They are not up to the challenge, says Jonny Dyer, who worked on a Google project that first brought the OneWeb team together (but stayed with Google when the two parted ways). “The supply chain does not scale,” he says. “They’re not used to working at those volumes, and they’re not used to the unit cost.”

OneWeb and SpaceX thus not only have to make new satellites, they have to build a system for building satellites. OneWeb has been doing so in Toulouse for the past two years. Its first satellite was completed in April and ten more will be ready in time for the company’s first launch, some time before February 2019. To step up manufacture, OneWeb is building two copies of its production line in a new factory in Florida. It hopes to have the first satellite from this facility ready before March 2019 and to raise output to ten a week not long after.

The factory floor in Toulouse has separate workstations for propulsion systems, communications payload, solar panels and so on. Satellites in the making move on robot carts from one station to the next. Cameras track the components and look out for errors—misalignments and the like. The finished cube is about the size of a beach ball bedecked with antennae and solar panels. After testing, it is shipped out. The system has had teething problems. The first launch will be more than a year behind schedule. But Greg Wyler, OneWeb’s boss, says he still hopes to offer connectivity in places in higher northern latitudes, such as Alaska and Britain, by the end of 2019.

Putting satellites in place is only part of the problem. How useful they will prove to be depends on designing and building antennae to get data to homes or vehicles that are not close to terrestrial cell towers. “The elephant in the room…has always been the ground terminal,” says Nathan Kundtz, the former boss of Kymeta, which makes antennae. Mr Kundtz says that tracking satellites across the sky mechanically is untenable if the antennae are to be affordable and widely used. His firm does tracking electronically. No moving parts, he says. Teledesic failed in part because no such ground terminal existed in the late 1990s. Fortunately, the necessary electronics have shrunk in size and cost.

Aerial combat

Firms such as Kymeta, along with at least two other companies, Phasor and Isotropic Systems, are producing flat, electronically “steerable” antennae with no moving parts that can send and receive signals from leo satellites. Kymeta’s antenna is the least orthodox. It relies upon the same kind of lcddisplay found in laptops and flat-screen televisions. Instead of using the 30,000 pixels in its screen to display images, it uses them to filter and interpret the satellite signal by allowing it to pass through at some pixels and blocking it at others. Different patterns of pixels act like a lens, focusing the signal onto a receiver beneath them; the pattern shifts up to 240 times a second, changing the shape of the “lens” and thus keeping track of the satellites overhead. Phasor’s system works similarly, but uses an electronically controlled array of microchips to perform the same task. Isotropic Systems, which has said that it is developing an antenna that will be able to receive signal from OneWeb’s satellites, uses an optical system more like Kymeta’s.

Kymeta and Phasor have both said that they do not want to sell antennae directly to consumers, but will focus on businesses, cellular networks, maritime and aviation customers instead. Isotropic Systems has announced that it will use its technology to produce a “consumer broadband terminal” in time for OneWeb’s launch. Once available, consumers are most likely to get the new pizza-size antennae through their internet service providers. But if it is too expensive for people to receive signals on the ground—most of the world’s unconnected are poor—those ventures selling direct to consumers will struggle. Mr Wyler says his firm needs antennae that cost $200 at most for the consumer business to thrive.

Telesat, the next biggest firm in terms of constellation size, is taking a different approach. It does not plan to offer services to consumers directly, but instead is focusing on filling in gaps for cellular networks, as well as businesses, ships and planes. Specialised telecoms companies would buy bandwidth and resell it. In contrast to Messrs Wyler and Musk, and their aspirations for global coverage, Telesat has divided the surface of the planet into thousands of polygons, and modelled exactly in which ones it makes financial sense to offer strong connectivity. This means its constellation needs fewer expensive gateways.

Mr Wyler, in contrast, is known as something of a connectivity evangelist. His first satellite internet firm, o3b (Other 3 Billion), placed large satellites in a higher orbit, providing a connection only slightly slower than a leosatellite. Now owned by ses, a larger satellite company, o3b specialises in providing connectivity to islands that are otherwise cut off. OneWeb’s goal of connecting consumers is largely in the hands of SoftBank, its main investor, which owns the exclusive rights to sell the new bandwidth.

Even if the new satellites bring the internet to people and parts of the planet that have been ill-served up until now, putting ever more objects in space brings another set of difficulties. Satellites in densely packed constellations may crash into each other or other spacecraft. “If there are thousands [of satellites] then they’ll have much higher probability of colliding,” says Mr Dyer. “If there is a collision in these orbits it will be a monumental disaster. At 1,000km, if there’s an incident it will be up there for hundreds of years.” Geostationary satellites, because they do not move relative to each other, are unlikely to collide.

Managing constellations is particularly difficult, says Mr Wyler, because each satellite has only a tiny amount of power to work with (equipping small ones with bigger thrusters would be hugely expensive). So even if a crash were imminent, there would be little that could be done about it other than watch. “What are you gonna do? Nothing. Get popcorn. There’s nothing to do,” says Mr Wyler. OneWeb has designed its constellation so that faulty satellites fall out of orbit immediately to avoid this risk.

Access all areas

The new constellations will also raise tricky questions of national jurisdiction. Countries generally have control of the routers which connect them to the wider terrestrial internet. Satellites threaten that control. The national regulators that OneWeb has talked to are uneasy, says Mr Wyler, because it would create a route to the internet that countries could not monitor. OneWeb’s intention is to build 39 “gateways” on the ground around the world that will beam up and receive traffic from its satellites.

The first is under construction in Svalbard, a remote Norwegian island chain. These access points, and those planned by other firms, present another difficulty. Some countries are willing to share gateways with other countries. Others want their own because they are concerned that third parties will be able to monitor internet traffic, potentially using it to hack data flows of national importance.

Questions remain about whether the businesses involved can do all they promise cheaply enough. But if these companies succeed, their impact will go beyond helping to bring 3.5bn people online. Mr Musk has hazy plans to use Starlink as the foundation for a deep-space network that will keep spacecraft connected en route to Mars and the Moon.

With a network of satellites encircling the planet, humans will soon never be offline. High-quality internet connections will become more widespread than basic sanitation and running water. The leo broadband firms are trying to reinvent the satellite industry. But the infrastructure they are planning will provide a platform for other industries to reinvent themselves, too.

Correction (December 11th, 2018): This piece originally stated that Intelsat has launched 59 satellites in its 54-year history. That is the number of active satellites the firm has in orbit. The firm has successfully launched 94. Sorry.

This article appeared in the Briefing section of the print edition under the headline “A worldwide web in space”

Big Data, The Cloud And Smart Mobile Are Actually One Big Thing


ToDaClo is a current buzz word of sorts for “touch-data-cloud,” (or Big Data, The Cloud and Smart Mobile)  which appears to have been coined by a writer for Forbes magazine during a talk in Paris in May 2012.  The speaker declared the death of the previous buzz word, SoLoMo (social-local-mobile). ToDaClo does not seem to have caught on beyond France as most of the writing and blogging about it is in French.  SoLoMo had a following for some time, and even has an online manifesto, vaguely implying location based services, which have been a major mobile feature for some time in Asia, but not here.  I think the bottom line is that both of these acronyms are trying to communicate the concept that Big Data, The Cloud and Smart Mobile are inter-related.  I actually think of them as One Big Thing, even The Next Big Thing, or perhaps “Ne-Bi-Ng”  (Nebing) as some may prefer, though I doubt Nebing will ever catch on.  Sanjay Poonen, President & CEO of SAP also views them as One Big Thing.

Reblogged from Gigaom

The secret to tackling mobile, cloud and big data? Treat them as one.

by Sanjay Poonen
sanjaypoonenSAPSanjay Poonen, President & CEO of SAP AG
SUMMARY:It’s no secret that mobile, big data and cloud computing are transforming IT. Sanjay Poonen, president of SAP’s mobile division, says companies need a single unified strategy to tackle them, not three separate ones.

There is widespread agreement—across the globe and in every industry—that mobile, big data, and cloud computing are the three cornerstone issues of tomorrow’s business environment. In fact, a strong organizational response to each of these issues is already critical to competitive survival.

As a result, CIOs, business strategists and IT leaders are working furiously to make sure their businesses have plans in place to stay ahead of these challenges. But there is one subtlety that is frequently overlooked: When it comes to mobile computing, big data and the cloud, what we have is not three problems but one.

Rising in unison

It’s not a coincidence that the profile of these three business challenges rose in parallel. Mobile, big data, and cloud are not siloed concerns easily addressed in isolation. They exist in an overlapping matrix, where the importance of each issue increases because it leverages (or helps solve) an issue raised by one of the others.

For example, in the days before mobile computing, business users typically did all their work using just a handful of applications. Today, the average smartphone has 41 apps installed on it. And each of those applications sparks a need to consider security, since it generates data each and every time it is used. And because these devices are often connected to service provider networks – rather than directly with corporate servers – a great deal of that business app data requires secure cloud storage.

Thus the proliferation of mobile devices exacerbates the big data problem, which in turn precipitates the demand for cloud.

In short, they are all part of a single, converged and symbiotic trend. And to address them optimally requires a holistic perspective on all three.

No bottom in sight

With global demand for mobile computing at the heart of this escalation, it makes sense that IT strategists would be keenly interested in the trend lines for mobile adoption. Today, 87 percent of the world’s population owns a mobile phone; 60 million Android devices were sold in the second quarter of 2012, and now 1 million new Android devices are provisioned daily, according to Google. As of last month, there were likely more smartphones on the planet than humans, according to Cisco.

So the question is whether there is a saturation point on the horizon that could help curb this cloud/mobile/data demand? Surprisingly, no. The average number of mobile devices per employee worldwide has already reached three to five, and adoption rates continue to grow as consumers add tablets and ever-more capable smartphones to their mobile arsenals.

But consumers’ ceaseless enthusiasm for new form factors and functionality is not the whole story behind the world’s bottomless demand for mobility. Today, businesses themselves – rather than consumers – are adding fuel to the fire.

Not just a BYOD issue

As industries finally crest the hump of transforming their workflows to leverage mobile device availability, they drive new demand – not only for mobile devices, but for new scalable infrastructures that deliver more actionable intelligence from their big data.

Finance Consumer banks, operators and retailers are widely deploying mobile commerce capabilities, which, in addition to automating traditional transactions, must include on-demand access to unstructured data, such as check images.

Manufacturing  Mobile devices on the factory floor automate manual processes, thereby feeding more rapid information into the system. This makes it possible to detect and respond early to issues that take a toll on quality or productivity, such as supplier errors.

Retail  Retailers are giving regional store managers mobile app access to daily and even real-time sales performance data on the floor, allowing them to optimize displays and customer service to sell more of the most popular items.

Health care Thanks to new mobile apps and devices, the details of every patient interaction is now entered into the system nearly instantaneously. This provides a basis for a more efficient and orchestrated care response, and in some cases leading to more rapid or accurate diagnoses.

The internet of things

As mobile technology embeds itself into more and more objects, vehicles, buildings, sensors and machines, the heterogeneity of actionable business information will only grow. “Annual global IP traffic will surpass the zettabyte threshold by the end of 2016,” reports Cisco. “In 2016, global IP traffic will reach 1.3 zettabytes per year or 109.5 exabytes per month.” (As we already know, there are currently at least 2.7 zettabytes in storage globally).

Smart equipment and vehicles will upload data to service provider networks as well as private networks, and organizations will need a plan to normalize data in many forms and from many sources. The scalable infrastructures we design today to store and structure such varied data are critical to the enablement of the business innovations we will need in the future.

The effect of this convergence is already apparent, especially in the area of business intelligence. Mobile business intelligence makes it possible for organizations to provide analytics on key performance metrics to a wider variety of employees – not just for executives. Once employees get a taste for how mobile apps fuel greater effectiveness in their job duties, they will push for more dashboards and more data. And these big data stores can’t be undertaken without cloud, to facilitate real-time performance, nor mobile devices and apps, to deliver data into the field where it’s put to good use.

Embracing the Entanglement

The interdependence of mobile, big data and cloud is undeniable, and will only multiply as data growth and mobile use continue. Yet our strategic thinking lags behind the evidence. As we have learned from IT revolutions of the past, a partial strategy is worse than no strategy at all, as you can end up with an inflexible, tactical implementation that requires a ‘rip and replace’ approach.

Organizations that manage to avoid a false start with a siloed strategy will create a network design better aligned with where IT will be in five years. In short, the most successful organizations recognize the secret alliance of mobile, big data and cloud early, and develop a holistic strategy considering all three in concert.

Big Data: Big Deal Or Not?


I have been having a spirited marathon debate with a couple of my friends.  Is this alleged new “Big Data thingy” so transformational that it will change our every day lives, or is it just an evolutionary advance that may improve productivity but not much else?  The same arguments may apply to the concept of “The Cloud,” and “Smart Mobile.”  The three, taken together, are coalescing into the major information technology forces that will drive innovation and productivity into the foreseeable future.

PollDaddy: What Is Your Opinion?  Big Data: Big Deal Or Not? Or Comment Below

We are hearing regularly in the media about so-called “Big Data.”  What exactly is Big Data? A number of differing definitions have been offered from a wide range of media sources. ZDNet‘s definition is one of the best I have seen so far.  In essence, big data is about liberating data that is large in volume, broad in variety and high in velocity from multiple sources in order to create efficiencies, develop new products and be more competitive. Forrester puts it succinctly in saying that big data encompasses “techniques and technologies that make capturing value from data at an extreme scale economical”  Prior to the emergence of commercial Big Data, the concept only existed where cost was no object: in the black world of the National Security Administration, and required the largest purpose-built supercomputers in existence.

bigdatalandscape

zettabyte (symbol ZB, derived from the SI prefix zetta-) is a quantity of information or information storage capacity equal to 1021 bytes or 1,000 exabytes (or one sextillion (one long scale trilliard) bytes).[1][2][3][4][5]…..I Billion terabytes….Today, you can walk into your local computer store and buy a couple of terabyes for a $100.  Only $500 Million for a zettabyte.  In real terms that is dirt cheap, and getting cheaper daily.   Now that we have that cleared up, we can move to the next level.

With regard to the obvious issue of personal privacy, the European Union and other organizations have made efforts to protect privacy, with very mixed results.  Other governments, notably China, are aggressively implementing opposite policies to strictly limit privacy.  Highly sophisticated telecommunications equipment has been available for years that enables deep analysis of all of your voice and Internet traffic. We learned this when Dick Cheney secretly set up such equipment to track and record all voice and data traffic in the United States.  The equipment trapped and analyzed all of it in real time. You didn’t notice a thing.  The thing about your personal data is that they already have it. Most of it comes from public sources you authorized.   I not advocating this, I am only the messenger. The founder and former CEO of Sun Microsystems, Scott McNealy famously said, “You have no privacy. Get over it.”  We must not ignore the serious issue of privacy, but the problem is already here and deep data mining is thriving.  Privacy needs a revolution of its own.

The core question then becomes whether Big Data, and for that matter, the Cloud, and Smart Mobile, represent revolutionary and transformational changes in technological capability and also consequentially, human culture, politics: how we conduct ourselves in the World.  Or is it just so many more boring zeros and ones zooming by at the speed of light, stored in chips, and processed by quantum microprocessors?  No big deal, just IT management as normal.  Frankly, this is a significant philosophical question.  For this discussion, we will focus only on Big Data.   Discussion of the Cloud and Smart Mobile will follow later.  My most recent post on Smart Mobile gives a hint of my thoughts:  Mobile Market Share: A War of Titans Worth Following, http://mayo615.com/2013/01/21/mobile-os-market-share-strategy-war-of-the-titans-worth-following/

In fairness, I cut my teeth on Marshall McLuhan‘s ideas while in university in the 1960’s.  In an amazing irony, I soon fell into Intel Corporation at the birth of the microprocessor revolution, and later, I was also present to personally participate in the emergence of the personal computer. My memory of McLuhan kept popping up everywhere.   As my career progressed, I seemed to jump onto each new wave: networking at Sun Microsystems,  then the Internet infrastructure build out explosion with Ascend Communications, and finally a host of new companies, based on Internet-based capabilities.  Through all of it, I could only conclude that somehow McLuhan, like some kind of modern Nostradamus, had foreseen it all.   Most importantly, my own life was transformed by it all, and I saw with my own eyes the massive transformation occurring all around me.

globalvillage

So I have no doubt that Big Data is transforming our lives, and will continue to transform our lives, in ways we cannot yet fully grasp, as I could not grasp McLuhan when I first heard him, or the significance of the Internet as I sat right in the middle of it.

I have previously described Big Data as analogous to the evolution of Chaos Theory.  For centuries, full understanding of the complexity of nature’s designs were thought to be the realm of God, and beyond human comprehension and explanation.  Then in the 1960’s in places like Santa Cruz, California and Germany, the elegant simplicity of a solution to chaos began to emerge.  The massive scale of Big Data is a very similar nut to crack. We are now seeing an elite group of data scientists and mathematicians begin to solve Big Data in a way similar to how chaos was resolved.  Google, Microsoft Bing, Baidu, Yahoo and Amazon are driving the development of these mathematical skill sets.

chaos

Last year I showed my UBC Faculty of Management students a YouTube video on Data Mining. In the video, the two Hungarian mathematicians leading a data mining company, described how they had solved hideously complex problems that were previously beyond any computational solution. The key to their success was their ability to extract very precise useful information from extraordinarily large stores of information.  The metaphor here is more like finding a particular grain of sand on a very large beach.  A parallel key factor has been the incessant march of Moore’s Law.  Even 10 years ago, successful data mining on this scale could not have been accomplished. The computational cycles and high speed mass storage were not available or were too expensive.   Today those microprocessor cycles are available.  The costs will continue to plummet, making further advances inevitable. Failure to consider Moore’s Law and available computational cycles has also been the cause of many failed ideas over the years. But the threshold has arrived.

Today, developments like Google Spanner, the largest known database architecture in the World, have joined with the computational solutions.

Unveiled this fall after years of hints and rumors, it’s the first worldwide database worthy of the name — a database designed to seamlessly operate across hundreds of data centers and millions of machines and trillions of rows of information.

Spanner is a creation so large, some have trouble wrapping their heads around it. But the end result is easily explained: With Spanner, Google can offer a web service to a worldwide audience, but still ensure that something happening on the service in one part of the world doesn’t contradict what’s happening in another.

google-spanner

Google’s decision to reveal Spanner has many dimensions.  First, it provides a peek into the black World of the U.S.  National Security Agency and the U.S. Defense Intelligence Agency.  Previously, the existence of such large and sophisticated global databases were only imagined. We now know they exist and are a crucial component of Big Data.

Read more in my post, Google Spanner, the single largest database in the world

http://mayo615.com/2012/11/26/inside-google-spanner-the-single-largest-database-in-the-world/

For me, the most compelling example of how this all works, has been the extremely sophisticated Big Data mining used by the Obama campaign to achieve re-election. As early as March 2012, the Wall Street Journal began reporting about “Dashboard,”  the Obama campaign app that was mining Big Data to find undecided voters in key states.  But not only undecided voters.  Dashboard can key in, find and persuade “Off the Grid” voters.  Off the Grid is the term used to describe those people, such as students and other young people, with constantly changing locations and only a mobile phone.  These voters have historically been virtually impossible to reach.  This short PBS Newshour video below speaks volumes about the extraordinary impact and value of Big Data, not seen before.

Watch How Much Do Digital Campaigns Know About You? on PBS. See more from PBS NewsHour.

The campaign’s hiring of Rayid Ghani, as “chief data scientist,” and an army of data analysts, set the stage for what was to come.  On election night, Mitt Romney and Paul Ryan were absolutely convinced that they had won the election, but were shocked to find otherwise. Working through their disbelief, both candidates later remarked about the enormous voter turnout for Democrats in key locations and the “technology advantage” of the Obama campaign.

So from my years of observation of the march of technology and its impact on my own life, I am convinced that we are entering another transformational period as profound as the emergence of the Internet itself.

I have been repeatedly drawn back to Steve Job’s 2005 Stanford University commencement address, in which he closes with references to Stuart Brand and The Whole Earth Catalog. Stuart Brand is an extraordinary futurist.  One of Ken Kesey’s original Merry Prankster’s chronicled in Tom Wolfe’s book non-fiction novel, The Electric Kool-Aid Acid Test, Brand had been inspired by the legendary first photograph of the entire Earth taken by Apollo 8 astronaut Frank Borman.  Brand is also the founder of The Well,  the very early Sausalito-based Internet Service Provider, who is now considered one of the most important thinkers on human culture, technology and its impacts.  Word of Job’s commencement address spread virally around the Valley...”Did you hear what Job’s said at Stanford today?”    Steve was basically saying that he too understood what McLuhan had said, and that Stuart Brand also understood the transformational importance of the Global Village, by publishing The Whole Earth Catalog.

stuartbrand2

WholeEarthCatalog