The Web 2.0 Revolution (5) Search 3.0

Search engine itself is a product of Web 1.0; it is so not Web 2.0.

---

[+] Search is a product of Web 1.0

In my last article, I talked about the business paradigm for Internet media and online advertising brought forth by Google. It is about "many a little makes a mickle," which is very Web 2.0 in every way. Now it looks like Google will become the new paradigm which is going to last happily for ever. Yet, is it really so?

Let's not forget that search engine is a product of Web 1.0 and has been there for over ten years. The logic of search engine is to have everyone connected to a certain website, where there is a central host that does all the computation and comes up with search results. This is so not Web 2.0.

In addition, although Google has managed to enhance the accuracy of keyword search to an extraordinary degree, very often users still get a chunk of irrelevant results. The next generation search technology thus becomes the target eagerly pursued by all major Internet giants.

Many people, however, think that the weak link of technology can be compensated by "social search," as Web 2.0 is characterized by "sharing and participating by the public." After all, computers are driven by programs or a fixed logic, which can never keep up with the flexibility of human brains. As such, it seems to make sense to improve search accuracy by human assistance.

A practical solution is to let an Internet user define the keywords (the so-called "tags") for her/his own content when posting articles or pictures. This will make it easier for other people to find the user's content when they search by the same keywords.

[+] It is the search cost that matters

This is what is called "social search." By having people define the keywords or tags for their own content, and search engine index and compute these tags, the degree of accuracy can be further enhanced. After all, these keywords are defined by human brains and may be more likely to fit in with other people's way of thinking.

This search method - we may call it Search 2.0 - may not appear very relevant to technology. It is also the result of the ever-declining storage and bandwidth cost, which makes it possible for everyone to produce content on the Internet. Ten years ago, it was unlikely for such a search method to emerge and diffuse, let alone to become the mainstream.

Yet "social search" is not the ultimate solution. In fact, practically all search methods will have to face the problem of cost in addition to accuracy. This problem can be roughly represented by the following equation:

[+] Every search you make online costs money.

"Every search you make online costs money." In the past few years, the explosive growth of searches has been driving search engine operators to purchase new equipment and to expand the space for settling such equipment from time to time, so that operators can provide accurate and fast search service.

From the above equation, we can see that if an operator fails to control its cost properly, the average cost per search will increase. However, if reduction of cost will sacrifice the performance of accuracy and response time, the average search cost will drop but users may start to leave.

[+] Towards Search 3.0

Thanks to the decreasing storage and bandwidth cost on the Internet, equipment is not as costly as before. Hardware size has been shrinking, which has made it possible to accommodate more appliances in the same data center. Therefore, the cost of data center may be economized and the financial pressure on operators may be somewhat relieved. However, in spite of the many efforts, it has been very challenging to drive the computation cost down.

Google is one of the search engine websites which has a clear understanding about this point, so it has attended to the cost of equipment very carefully. It rarely purchases host computers from external vendors - tens of thousands of its servers were designed and developed by it and assembled from low-cost PC components. That is how it keeps cost down.

Still, Google needs to deploy data centers in order to provide better quality of service. Its new Oregon data center, built along the bank of Columbia River, is as big as the area of two football courts, and has all sorts of hosts and servers.

There is nothing wrong about such way of thinking; it is only very Web 1.0. No matter how deep the pockets of search engine operators can be, it can only own a limited quantity of servers. Be it ten thousand or one million, it can only be fraction of the hundreds of thousands of PCs globally.

I would like to reiterate the spirit of Web 2.0: "many a little makes a mickle." When a search engine user clicks on "Search", which mode do you think will be the faster to get what s/he wants? Through the Oregon Data Center in the U.S., or through a huge number of PCs all over the world?

[+] P2P search: the next generation search method

In fact, Google has developed many technical innovations including parallel processing, which is to divide a single query into several ones to be sent to several servers. This technology may have been applied to Google's search engine. Every search you make may involve several servers.

The question is, when will the above mentioned "several servers" become "all PCs connected to the Internet globally." Instead of Google's host, every search of yours involves thousands of home PCs, like the one you are using now, at different households the world over.

If one day the impact of technological innovations and the ever declining average cost of home broadband access and storage worldwide have got to a certain critical point, the existing search method on the Web may be overturned! Due to the technological bottleneck and security concern behind such expectations though, it is not easy to have significant breakthroughs in the short term.

In addition, there is also an economic problem: why do I need to offer my PC and home broadband connection to process your search? Maybe "one day I may need your help with my search" is a good reason, which sounds even more advanced than Web 2.0. We may as well call it P2P Search.

With the continual reduction of the cost of online storage and broadband access, will there be one day when it is cheaper to throw a query to PCs worldwide than to proprietary servers? This is an issue that search engine operators on the globe need to think over seriously. (
2006/09/24 - By Digitalwall.com - Way to
China Internet/Telecom )

---

- Read More

Prev : The Web 2.0 Revolution (4) the Google Paradigm


Next : The Web 2.0 Revolution (6) Struggle of the Press Industry

---

- Today in History

The Web 2.0 Revolution (5) Search 3.0 - 2006/09/24

Crime and Punishment of P2P (1) Liberalization of Power - 2005/09/25

Three Musts of Digital Content Biz (3) Redefining Ownership - 2004/09/26

Corporate Website a Handful (1) Accountability Where? - 2003/09/28

The Web 2.0 Revolution (4) the Google Paradigm

Google does not expect to be the biggest pageview generator in the world, because it knows that most pageviews happen outside Google.

---

[+] Google profits without being a monopoly

In February 1999, I brought up with the rule: for a media company to make a profit from advertising alone, it must be a monopoly or oligopoly; yet no one can monopolize the Internet, so finding other revenue sources other than advertising is a requisite. My argument at that time was based on my reflection on the nature of the media per se, though few shared the same with me.

So far my idea has proved true, but still I've missed a fact: the energy derived from declining storage and bandwidth cost on the Internet and the continuing advance of technology has continued to build up to a degree so overwhelming that it starts to transform the cost structure of the Internet media business and to overturn the old rule.

"One need not monopolize to make money from advertising," - this is one new paradigm brought forth to the world by Google. And here's another - "there is no need for users to stay on search engine websites; they'd better leave as soon as they've found what they want."

Ten years ago search engine operators worried about the low average pageviews per visitor, so they came up with numerous services in the hope that users would stick longer so as to increase pageviews and ultimately more advertising sales. Yet what Google has been doing is contrary to this thinking, which is very much like a relentless blow to the old-minded operators.

If we look at Google's tricks, we will find that, on the technology side, there are two key factors: raising search accuracy and lowering average search cost (few people talk about this); on the business side, there are also two factors: keyword advertising - AdWords - and profit sharing with other website operators - AdSense.

[+] Google's very Web 2.0 business model

As mentioned in my previous articles, the fetal weakness of the Internet advertising business lies in the declining average advertisement price. There are two main reasons: exploding web pages divvy buyers' advertising budget and the embarrassingly low click through rate (CTR) is unattractive to buyers.

However, when search engines start to display advertisements based on users' search keywords, the CTR goes up dramatically to above 5%, an uplifting figure only seen during the Internet start-up period a decade ago. For buyers it means the improvement of advertising effects, which consequently leads to the rise of advertisement price.

This business model was not an original idea of Google, but Google is able to make the best use of this model with its extraordinary technological capability and unparalleled search accuracy widely recognized by Internet users. Google has attracted the highest search volume and thus obtained the most advertisement clicks.

However, few people would come across the idea that Google's advertisement approach is very Web 2.0. One of the major features of Web 2.0 is "many a little makes a mickle;" small forces can add up to a big change. In fact, keyword advertisements appeal mostly to small and medium enterprises (SME's).

This is very different from the approach of traditional media (or traditional Internet media) which focus on alluring major buyers with big advertising budgets. Google instead looks at all SME's in the world, whose advertising budget spending in total can be no less than that of big advertising buyers.

[+] The cost is low to do business with a large number of small clients

Imagine you own an advertising agency company, and you want to do business with small companies worldwide. Considering the difficulty and cost that might occur, at the end you would only deal with a couple of rich clients as it is the most cost effective.

Yet again we see the magnificent result caused by the ever-declining Internet storage and bandwidth cost: it becomes acceptable to do business with SME's around the world! It is perfectly possible to grab big profits at a lower cost - an idea that was totally beyond imagination in the past!

Furthermore, we see advertisements provided by Google on other websites from time to time, too. This is Google's AdSense service working here. It allows other websites to display advertisements sold by Google and to share the advertising sales profits.

In fact, there are many small and medium sized websites over the world which makes considerable money by displaying Google's advertisements and sharing the advertising revenue. Once again, Google takes in the traffic on small and medium websites around the world and amasses a volume comparable to that of large-scale portals.

Google can profit without snatching a dominant share of major advertisers' budgets. It can make money without monopolizing online traffic. It does not expect itself to be the website which generates the most pageviews in the world, as it knows most traffic flows outside Google. To this point, the Internet media paradigm is completely overthrown.

[+] The "Long Tail" paradigm

The figure above is inspired by the Long Tail theory which has been very popular these years. The Long Tail theory was first brought up in 2004 in a US magazine Wired by Chris Anderson, who talked about the phenomenon of mainstream meltdown from Amazon's list of sales items - he found a great many of them were not in very popular demand.

Later, Chris Anderson continued to elaborate his theory by describing possible future economic patterns and these articles were compiled into a book and published in 2006. However, in my opinion, the Long Tail theory has been realized by Google since long time ago, and what has been manifested here is a true Web 2.0 spirit. Please allow me to reiterate that:

The root of the Internet revolution is but one thing - the ever declining cost of digital storage and transmission bandwidth. Socially, it is reflected on "the continuously falling cost of interpersonal communication; on the business side", it is "the gradual disintegration of enterprises which used to thrive on their monopoly of capital and information".

Our intention here is not to praise the vision or greatness of Google, but to contemplate on one thing: while the above mentioned cost continues to drop, those who can makes the best use of this trend to reach the most businesses and users will be able to get hold of this revolutionary power. So, are there any other business models yet to be developed? (
2006/09/17 - By Digitalwall.com - Way to
China Internet/Telecom )

---

- Read More

Prev : The Web 2.0 Revolution (3) Advertising Revenue is Not Enough


Next : The Web 2.0 Revolution (5) Search 3.0

---

- Today in History

The Next Step for Web 2.0 (3) Collective Will Is the Cornerstone of Everything - 2007/09/16

The Web 2.0 Revolution (4) the Google Paradigm - 2006/09/17

The Web 2.0 Revolution (3) Advertising Revenue is Not Enough - 2006/09/10

Envisioning China's 3G Market (3) Systems & Markets - 2005/09/11

Three Musts of Digital Content Biz (2) Stop Selling "Containers" - 2004/09/19

Three Musts of Digital Content Biz (1) Content is Cheap - 2004/09/12

The Web 2.0 Revolution (3) Advertising Revenue is Not Enough

The rule of the game in the media business is monopolization, yet it is impossible to monopolize the Internet.

---

[+] The diluted advertising revenue

In February 1999, I wrote the first article to point out that portals could not survive only on advertising revenue (, which was contrary to the mainstream opinion at that time and has been proved true for the next five years until Google created a new model). My view was based on the following formula.

I guess you will agree with me that the growth rate of the denominator in the equation "Total Pageview" is much higher than that of the numerator "Total Ads Revenue". So you should be very clear about what is total advertising revenue divided by the number of web pages worldwide.

The conclusion is that the selling unit price of online advertising will continue to fall! When buyers' advertisement budget, which is usually fixed, is allocated to web pages around the world, each page can only get a slim portion of the money. This is exactly what the Internet is – a distributed network.

The cost of online advertisement is calculated on a CPM (cost per thousand impressions) basis, which is derived from the traditional media industry standard. In fact, the figure of CPM has been declining since 1999.

When is average unit price of online advertisement worldwide keeps falling, it becomes very dubious whether a website can survive solely on advertising revenue or not - especially when each of the websites continues to produce a vast amount of web pages to dilute the advertising revenue.

[+] The rule of the game in the media industry is monopolization

Gradually, the increasingly dire conditions start to press hard upon the portals. Judging from the above equation, there are only two ways to survive. One is to snatch the limited online advertising budget of the buyers and starve the competitors:

That is why we see major players take the initiative to lower their price. The formula shows that when the selling unit price drops, the websites with fewer visits will not be able to generate sufficient revenue to support their operation. In fact since 2000, there have been quite a few similar cases in which major operators voluntarily reduced their prices, causing smaller players to exit the market.

The second tactic is to attract all online users around the world to my website so as to maximize the number of pageviews. As long as the number of total pageviews is high, even the advertisement unit price is low, the total revenue accumulated can be phenomenal as well:

Either of the two methods points out one substantial fact that, only a monopoly or oligopoly on the global Internet pageviews will likely be able to survive solely on online advertisements. Yet, is it possible to concentrate all the pageviews in one single website?

This is what the media business is like. For all traditional media from newspapers to television, only a monopoly or at least an oligopoly will have a chance to enjoy profits. However, when the cost of storage and bandwidth on the Internet keeps falling, and the number of webpages globally continues to grow in an explosive rate, it is unrealistic to talk about monopoly.

[+] Low click rate and high cost of online advertising by community services

In addition to the above mentioned conflict between the characteristic of the Internet and that of the media business, there are also quite a few problems in the aspect of operation. Traditional media start to charge for the news content borrowed by Internet media to fill in the space. Traditional newspapers are not happy about the declining circulation and revenue. They now ask the users to pay for the content and to share the cost of content production, which is after all reasonable.

Another reason for the decline of the online advertising unit price is that the CTR continues to fall. Ten years ago, there were about 50 out of 1000 visitors who would click on the advertisements; now the figure is less than one out of 1000. The effect of online advertising has been weakening, which inevitably drives buyers to bargain on the price. You can imagine what the situation has become now after these ten years.

Furthermore, the quality of the content of online community services, such as chatrooms and forums, produced by users during their interaction, as well as the volume of traffic generated, does not live up to common expectation. There are junk content or deserted forums everywhere on the Net.

On the other hand, when users concentrate on their discussions and exchanges, they will not click on the advertisements. As such, the CTR of advertisements provided by online community services is naturally low. What is worse, services characterized by user interaction normally consume more bandwidth, which would result in higher operational cost.

As regards the free personal homepage services, they were very popular at the initial phase of Internet development – GeoCity was among the first and most famous providers. Yet from 2000 on, Yahoo! started to restrict the maximum traffic allowed for personal homepages. Once the number of pageviews exceeds the cap, unless users pay for the service, additional accesses to the webpages will be denied as as way to control bandwidth cost.

[+] A rule overthrown

The high bandwidth cost which characterizes online community services was indeed a problem in the Web 1.0 era. At the end of 2004, Yahoo! announced the termination of its chatroom service for these chatrooms were flooded with porn or junk content. Insiders knew very well that this was all about closing down a costly but profitless service.

Services such as free email accounts and free online calendars all have similar difficulties. For example, there are few people who would bother to click advertisements when reading emails. Yet such services are so important that no portals dare to call them off.

Since operators can't rely on advertising revenue alone, it then becomes necessary for portals, which accommodate a huge number of services, to create income streams other than online advertising. Such a strategic imperative has gained very high priority among portals since 2001, and Yahoo!, one of the most determined, has endeavored to raise the percentage of non-advertising revenue. To this point, my prediction has come true.

By 2003, these operators had developed multiple sources of income. For those which had survived the chilling winter, it was expected that the Internet business would move on with these portals leading the way. To everyone's surprise, there came Google coming up with a brand new rule. We should not have made light of the Internet at all. (
2006/09/10 - By Digitalwall.com - Way to
China Internet/Telecom )

---

- Read More

Prev : The Web 2.0 Revolution (2) the Emergence of New Media


Next : The Web 2.0 Revolution (4) the Google Paradigm

---

- Today in History

The Next Step for Web 2.0 (3) Collective Will Is the Cornerstone of Everything - 2007/09/16

The Web 2.0 Revolution (4) the Google Paradigm - 2006/09/17

The Web 2.0 Revolution (3) Advertising Revenue is Not Enough - 2006/09/10

Envisioning China's 3G Market (3) Systems & Markets - 2005/09/11

Three Musts of Digital Content Biz (2) Stop Selling "Containers" - 2004/09/19

Three Musts of Digital Content Biz (1) Content is Cheap - 2004/09/12