Microsoft Sharepoint Products and Technologies Resource Kit eBook

610 Part VII: Information Management in SharePoint Products and Technologies
You can create as many content sources as you need, but once you get above
a few hundred content sources, the user interface will become unworkable. However,
you can still manage this many and more by using the command-line parameters.
The SharePoint Portal Server object model is the recommended method of
working with content sources in large server farms with hundreds or thousands of
content sources.
In smaller installations, the more content sources you have, the more scheduling
you’ll need to manage because you won’t want all your content sources firing at
the same time to index their respective data. While it is not unusual for organizations
to have hundreds of locations from which they could potentially crawl content, it is
also conceivable that creating several hundred or even thousands of content sources
would present significant resource needs and (perhaps) administrative difficulties.
Therefore, you’ll need to balance several competing needs as you work with
SharePoint Portal Server. First, you’ll need to ensure that you crawl only data that
you really need in your index. For example, if you need to crawl 100 documents that
are located in a file share hosting 3,000 documents, crawling all 3,000 documents to
get the data from the 100 documents into your index would be foolish. Your index
would be filled with data that your users wouldn’t want appearing in their result
sets. Indexing needless information only clutters your index and the result sets, leading
to a less positive end-user experience when they use the portal site’s search
functionality.
So, what would you do? A best practice is to move these 100 documents into
their own file share and crawl that file share individually. Remember the old adage:
garbage in, garbage out. If you fill your index with needless information, the result
sets will not be tight and pinpointed toward what the user is really after. Keeping
your indexes clean and trim will help lead to a positive end-user experience when
they input a search query. Remember that the goal of creating content sources is to
enable your users to find information quickly and easily. A cluttered index leads to
a result set that forces your end users to hunt through the list to find the information
they are after. Such hunting is neither quick nor easy, and it’s likely to “turn off” your
end users from wanting to use the search features in the portal site. This is why, later
in this chapter, we’ll spend time looking at ways to craft the result set for your end
users.
Second, you’ll need to ensure that content sources are not created on a whim
or at every request from a user. It is conceivable employees from your sales department
will decide that they’d like to have a host of sale-oriented articles on a website
indexed for their own use. While SharePoint Portal Server can certainly handle this
task, it might not be wise to set up content sources at the request of each user
because you could end up creating many additional content sources that are really
unnecessary. A best practice to guard against this is to create a SharePoint Portal
Server planning team that lives in perpetuity and that approves the creation of new
content sources.