SEKE 2012 Proceedings - Knowledge Systems Institute

three different learning techniques: K-Nearest Neighbor (K-
NN); Naive Bayes; and Support Vector Machine (SVM). In our
experiment, we adopt the RapidMiner (http://rapidi.com/content/view/181/190)
tool to perform document
classification.
V. DATA SET
Our data set consists of structured hierarchy ontologies of
course syllabi of three universities; Cornell University,
University of Michigan and University of Washington. In order
to test our approach, we use three MASs as teacher agents:
Ag C , Ag M and Ag W . Each MAS controls a repository that
contains one of the three ontologies for the course syllabi of the
three universities. We also set up a learner agent Ag L to learn
some new concepts from the three teacher agents at the same
time. We choose to learn the concept “computer science”
because it has different representations among the three
ontologies. The University of Michigan organizes “computer
science” as an engineering discipline and as a joint program
with Electrical Engineering. The University of Washington
organizes “computer science” also as an engineering discipline
but independent from Electrical Engineering and as a joint
program with Computer Engineering. Cornell University
considers “computer science” as an engineering discipline but
independent from both Electrical and Computer Engineering.
VI. EXPERIMENT
In this experiment, we want to show how using social
networks can improve the accuracy of the learning process. We
follow the same strategy proposed in [17] in choosing positive
and negative examples. This strategy depends on the value of
sim(q spec , C best ), where, sim(q spec , C best ) is the ratio between the
number of examples that meet the entered query and the total
number of examples of chosen concepts C best . If sim(q spec , C best )
is greater than a specific threshold value (we set the value of
the threshold to 0.6), this concept completely represents the
concept to be learned. In this case, we can use any of its
examples as positive examples. The negative examples can be
chosen from its si blings (external negative examples). In this
case, the siblings’ examples are considered a good source of
negative examples because the two concepts have the same
parent which means they have some common features. At the
same time, they are two different concepts, each with its own
set of examples. So the examples of the siblings are considered
discriminating examples. If sim(q spec , C best ) is smaller than the
chosen threshold value (i.e. 0.6), that means C new overlaps with
the concept C best , which means that some of the examples of the
concept C best reflect the meaning of C goal but the others do not.
In this case, the returned documents contain only positive
examples of C new . The negative examples can be chosen from
the rest of the examples (internal negative examples).
The scenario of our experiment is as follows:
1. No “computer science” concept is defined in the
learner agent Ag L . The learner agent Ag L therefore uses
only keywords to search for the best matching concept
C best in the teacher agents’ ontologies. The keywords
used are “(computer science) or (program language)”.
At the beginning, the strengths of ties between Ag L and
all teacher agents (Ag C , Ag M , Ag W ) are the same.
2. Each teacher agent searches its ontology for the best
matching concept C best that has the highest value of
sim(q spec , C best ). Depending on this value, each teacher
agent picks sets of positive and negative examples.
3. Each teacher agent sends its own sets o f positive and
negative examples to the learner agent Ag L .
4. Ag L develops a new concept “computer science” in its
ontology based on these examples using machine
learning methods, and then calculates the accuracy of
the learning process in each case.
5. Calculate the feature vector of the newly learnt concept
“computer science” in Ag L .
6. Calculate the closeness between the feature vector
created for the new learnt concept and feature vectors
of each C best chosen by each teacher agent.
7. Depending on the closeness values calculated in step 6,
set the initial tie strengths between the learner agent
Ag L and teacher agents Ag C , Ag M , Ag W .
8. Repeat the learning process by using the same
keywords used before as well as the feature vector
created in step 5 based on the tie strengths calculated in
7 (repeat steps 2 to 5).
In order to measure the accuracy of the learned concept, we
use the confusion matrix to measure the proportion of true
results (i.e. true positive and true negative as opposed to false
positive and false negative). The overall accuracy is calculated
as follow:
=
VII. RESULTS
The first step in our experiment in learning the new concept
“computer science” is to find the best known concept C best in
teacher agents’ ontologies.
At the beginning, the learner agent Ag L does not have the
concept “computer science” in its ontology. We use only
keywords to learn this concept. Our keywords that describe the
“computer science” concept are (“computer science” or
“program language”). According to the ratio between the
number of documents returned by the search and the total
number of documents describing each concept, we can
calculate sim(q spec , C best ) for all concepts in each teacher
agent’s ontology. The concept with higher value of sim(q spec ,
C best ) is chosen as the best matching concept C best for each
teacher. Table I shows the value of sim(q spec , C best ) of the
chosen concepts C best from each university.
We choose the following concepts:
“Computer Science E” from Cornell University
“Electrical Engineering and Computer Science” from
University of Michigan.
“Computer Science and Engineering” from University
of Washington.
For all universities, sim(q spec , C best ) < 0.6 is chosen to be the
threshold, so we select the negative examples internally from
the documents of the chosen concepts. In this case, the strength
of ties between Ag L and all teacher agents Ag C , Ag M and Ag W
(1)
264