Microsoft Sharepoint Products and Technologies Resource Kit eBook

588 Part VII: Information Management in SharePoint Products and Technologies
Filters
When a file is created, it is formatted in the proprietary format of the application in
which it was created. In other words, a document created in Microsoft Excel is in a
different format than a document created in Notepad. SharePoint Portal Server must
convert the contents of each document to a generic format before it can place the
content into an index. The process of changing the format of the document is called
filtering.
The purpose of a filter is to remove the proprietary formatting while extracting
the text of the document as well as the document’s properties. When a document is
run through a filter, it is converted into an unbroken string of Unicode characters.
Filtering is run by the filter daemon process (Mssdmn.exe).
IFilters
The actual component that is responsible for knowing how to interpret the format of
files is called an IFilter (or Index filter). The gatherer uses IFilters for scanning documents
for text and properties and for extracting the text and metadata from these
documents. It also filters out embedded formatting while keeping information about
the position of the text within the document.
SharePoint Portal Server includes IFilters for the following: Microsoft Office
files, HTML files, Tagged Image File Format (TIFF) files, and text files. However, the
IFilter for Microsoft Publisher is included in SharePoint Portal Server Service Pack1.
SharePoint Portal Server accepts filters for other applications also. IFilters for popular
file types such as PDF can be obtained from the vendor. If the gatherer component
has no IFilter for a particular file type, it will attempt to extract the text of a
document using a generic filter called the Null IFilter.
If an IFilter is not available for a document type that you would like to include
in an index, another option is to contact the vendor of the application used to create
the document or to create a customized IFilter component by using the SharePoint
Portal Server SDK. An IFilter must be registered with the operating system and associated
with a file type before SharePoint Portal Server can use it in the indexing process.
If you obtain an IFilter from a vendor, these instructions should be included by
the vendor. The IFilter must be registered on the computer running SharePoint Portal
Server that is crawling that file type. File types are indicated by file extension,
such as .doc for a Microsoft Word document. Once an IFilter is registered and a file
type has an IFilter associated with it, documents of that file type can be crawled and
included in the index. If no filter is registered for a file type, only the file properties
are included in the index.
Filtering Limitations
SharePoint Portal Server has a limit to the amount of text data that it can filter from
a single document. This limit applies only to the text in the document. It does not
apply to graphics or any other type of content. By default, the limit is 64 KB of text