untangling_the_web

DOClD: 4046925
UNCLASSIFIEDNIiOft OlililelAL use Or~LY
Yahoo Site Explorer: The Yahoo Site Explorer website is still in beta; its goal is to
help users learn detailed information about a specific website:
The Yahoo search database contains detailed information about the structure of
the web. In addition to the web pages themselves, the database stores
information about links among pages, and uses that information (as well as
additional algorithms) to gauge the popularity of a given page.
Site Explorer gives you access to this information so you can learn about a site.
To explore a site, you submit a URL using a search box, just as you would for a
normal web search. You can then click links on the results page to see detailed
information.
The Yahoo Site Explorer will reveal all the pages in a specific domain, all the pages
in any subdirectory of a domain, and all the links to a domain. The main purpose of
the Site Explorer is to help webmasters improve the rankings of their sites, as
evidenced by the capability for sites to submit missing uris, the fact that Site Explorer
provides 50 results by default, its web services APls, and its ability to export the data
to a tab separated (TSV) file for further analysis. The initial response from the search
community has been lukewarm, but I like this new tool because'it simplifies learning
about a site and, unlike Google, Site Explorer provides all the links to a site (Which
Yahoo calls "inlinks") instead of a limited subset of links.
Let's examine Site Explorer from the point of view of a researcher instead of a
developer. Here's the Site Explorer result page for the uri [http://www.who.int]
showing all the pages in all subdomains of that website. The order is by the most
visited pages at the domain according to Yahoo's records about the page:
Rli~1' lls;
~J?8$t2J8: .!J2.I .I. ~~I~~>. n ~o/~~i~ _. _ _,.. ..__
S!>fJW F.«ges !rom: Afl slIlidomains1.0nlyIhisdomaID
"W,h,ltni;,r.,iAAl.:Wtt\Ejrp1,(UiV Hilp.
.P ig i,. ,~;:i;o~;;i ~tiui'2 sa.1 32:.·bj. ';"6 :
r, ~~%~:;::=il l~~\S 1
~, 'EJ. ! r i{lhti: !I\,~~ · . i~ . ~~ . !t'i'~1r; URI,
7. OM":f l arf!@ t~h;~.~:iil"1 !~;~l.~
\~Wi. .~h l))r;if~~.;,; 2~}:':Gg,.~ i· ;fii;-i !i firlki
UNCLASSIFIEDHFOR OFFIGIAL 1:J8E mlLV 105