AI - a Guide to Intelligent Systems.pdf - Member of EEPIS

304
KNOWLEDGE ENGINEERING AND DATA MINING
the knowledge engineer (a person capable of designing, building and testing an
intelligent system) and the domain expert (a knowledgeable person capable of
solving problems in a specific area or domain).
Then we specify the project’s objectives, such as gaining a competitive edge,
improving the quality of decisions, reducing labour costs, and improving the
quality of products and services.
Finally, we determine what resources are needed for building the system.
They normally include computer facilities, development software, knowledge
and data sources (human experts, textbooks, manuals, web sites, databases and
examples) and, of course, money.
9.1.2 Data and knowledge acquisition
During this phase we obtain further understanding of the problem domain by
collecting and analysing both data and knowledge, and making key concepts of
the system’s design more explicit.
Data for intelligent systems are often collected from different sources, and
thus can be of different types. However, a particular tool for building an
intelligent system requires a particular type of data. Some tools deal with
continuous variables, while others need to have all variables divided into several
ranges, or to be normalised to a single range, say from 0 to 1. Some handle
symbolic (textual) data, while others use only numerical data. Some tolerate
imprecise and noisy data, while others require only well-defined, clean data. As
a result, the data must be transformed, or massaged, into the form useful for a
particular tool. However, no matter which tool we choose, there are three
important issues that must be resolved before massaging the data (Berry and
Linoff, 1997).
The first issue is incompatible data. Often the data we want to analyse store
text in EBCDIC coding and numbers in packed decimal format, while the tools
we want to use for building intelligent systems store text in the ASCII code and
numbers as integers with a single- or double-precision floating point. This issue is
normally resolved with data transport tools that automatically produce the code
for the required data transformation.
The second issue is inconsistent data. Often the same facts are represented
differently in different databases. If these differences are not spotted and
resolved in time, we might find ourselves, for example, analysing consumption
patterns of carbonated drinks using data that do not include Coca-Cola just
because they were stored in a separate database.
The third issue is missing data. Actual data records often contain blank fields.
Sometimes we might throw such incomplete records away, but normally we
would attempt to infer some useful information from them. In many cases,
we can simply fill the blank fields in with the most common or average values. In
other cases, the fact that a particular field has not been filled in might itself
provide us with very useful information. For example, in a job application form,
a blank field for a business phone number might suggest that an applicant is
currently unemployed.