untangling_the_web

DID: 4046925
UNCLASSIFIEDHFOR OFFlelAL USE ONLY
~ define (search): AWSP has a much more robust set of search options, syntax,
and APls than other search engines and also permits the use of stored
(canned) queries; the AWSP "data store" contains text, html, music, video,
images, and more types of files.
~ process: users can search the entire Alexa data store and "are able to
process both the raw content and the metadata extracted by Alexa's internal
processes."
~ publish: the output of the search can be anything from one result to an
entirely new vertical search engine, for example a new video search engine
or a new search engine for automotive parts. Quite literally, "by making use of
these utilities, a user might introduce a great new search service to the world
with nothing more than a home computer.,,69
The costs are modest and are based on consumption (you pay for what you use and
not for a subscription or service contract):
$1 per cpu hour ($0.50 for reserved but unused hours)
$1 per GB/year of user storage
$1 per 50 GB processed
$1 per GB uploaded/downloaded
$1 for every 4,000 user-published web service requests
In case you're curious, Alexa has a long history. Now owned by Amazon, Alexa was
created by Bruce Gilliat and Brewster Kahle (of Internet Archive fame), and until now
has been both famous and infamous as the technology behind the controversial web
traffic and website statistics "What's Related" toolbar feature in both Netscape and
Internet Explorer. The new AWSP is actually integrated into Amazon's web services
platform, something no one has done before."
Simply stated, Alexa/Amazon are "renting" their huge database ("data store") to any
and all takers for a remarkably reasonable price and, what is more, offering detailed
69 Alexa Web Search Platform User Guide, Introduction: What Can I Do with the Platform?
(17 January 2007).
70 There is one example of something similar, which came to my and some others' minds. If you are
familiar with IBM's WebFountain and its proprietary implementations for specific customers, you may
see some similarities. WebFountain also spidered the web and then let IBM's customers run queries
against that data set in more sophisticated ways than simple querying (something akin to
datamining). However, the problem with WebFountain and its progeny was that IBM had to write the
programs, and thereby hangs a tale of woe. For more, I recommend Jeff Dalton's blog entry on this
topic (I think he nails it). Jeff Dalton, "Alexa Web Search Platform: IBM WebFountain 2.0," Jeff's
Search Cafe,
192 UNCLASSIFIEDhTOR OFFIGIAL l:ISE 9NLY