Introduction to a Field Linguist's Toolbox - Computational Linguistics ...

Introduction to a Field Linguist's
Toolbox
HTMD: From Corpus to Lexicon (II)
Sascha Griffiths
Bielefeld, 2005
Bielefeld University
How to Make a Dictionary/
Dokumentation Bedrohter
Sprachen
Prof. Dr. Dafydd Gibbon
Winter Term 05/06

What is Toolbox?
●
Toolbox is a computational tool developed by the
SIL International (formerly known as the Summer
Institute of Linguistics)

What is Toolbox?
●
●
Toolbox is a computational tool developed by the
SIL International (formerly known as the Summer
Institute of Linguistics)
Toolbox is designed for field work purposes

What is Toolbox?
●
●
●
Toolbox is a computational tool developed by the
SIL International (formerly known as the Summer
Institute of Linguistics)
Toolbox is designed for field work purposes
Toolbox is a database application that
interlinearizes, analyses and stores text and can
convert this into an alphabetically ordered
dictionary

Interlinearize
●
“Interlinear Text Language data in tiered
format incorporating 'text, phrase, word and
morpheme levels.'
(Bird, Bow, Hughes, 2003) “ 1
1 From: http://emeld.org/school/glossary.html#interlinear 11/12/05
20:45:34

History of Toolbox
●
Original Name: Shoebox

History of Toolbox
● Original Name: Shoebox
● Why was it called Shoebox (up to version 5.0)?
– “The name "Shoebox" is a nostalgic
reference to the pre­computer times
when linguists used to store cards with
language examples in old shoeboxes.” 2
2 From: Nevskaya, Irina: Lecture Tutorial: An Introduction to Toolbox and
Shoebox (Abstract), on:
http://titus.fkidg1.uni-frankfurt.de/curric/dobes/sslectut.htm 11/12/05 17:47:58

What can Toolbox do?
●
“Shoebox is not a database system, nor an
information retrieval system, nor a text
corpus database in the strict sense. But it
combines all these aspects in a particular
way, which eases the work of the linguist
who does descriptive and analytical work.”
From: Hirzel, Hannes: How to Optimize analysing an African language text
corpus by exploiting old and new features of the Shoebox 5.0 interlinearization
program, on: http://www.unizh.ch/spw/tools/shoebox/LeidenCALL2001HH.pdf
11/12/05 18:05:30

SIL International lists it's Toolbox
under the following headwords:
●
●
●
●
●
concordance
database
dictionary
field notes
interlinear text analysis

SIL International lists it's Toolbox
under the following headwords:
●
●
●
●
●
concordance
database
dictionary
field notes
interlinear text analysis
You might want to google these
terms and later include these in the
portfolio

What SIL International writes about
Toolbox:

Basic functions
●
●
●
●
Viewing and Searching
Browsing
Editing
Sorting

Viewing and searching

Viewing and searching
Click on these arrows to view the
next/previous dictionary entry
(“record”)

Viewing and searching

Viewing and searching
Enter a word and press 'OK'

Browsing
Press Alt-R

Editing

Editing

Editing

Editing

Editing

Sorting

Sorting

Getting started
●
●
●
●
First download Toolbox 1.4 from
http://www.sil.org/computing/catalog/index.asp#software
Then download Toolbox Training from
http://www.sil.org/computing/toolbox/downloads.htm
Open Generic Start Up Kit
Then follow instructions as following

Before starting...
●
●
●
●
Copy the file 'Dictionary&Text.prj' into your
working directory
For this purpose click on the file with the right
mouse button (don't use drag & drop) and choose
'copy' or simply click on the file and press ctrl-c
Go to your working directory and click right
mouse button and choose 'insert' or press ctrl-v
Rename the file

The Startup Kit:

Starting
●
●
●
Select database and choose properties
Either set to English or vernacular
After that delete all the fields one does not need
for the first steps:
– Free translation
– Notes
– Morphemes
– Gloss
– Part of Speech

Entering text
●
●
●
●
●
●
Choose text as a source (example 'if')
Mark text in source document either using the
mouse or cursor
Press ctrl-c
Go to text row (\tx) and paste (ctrl-v) the text into
the right column
Enter the title of the text into text identification
Enter an abbreviation of the title into reference
line(the references can later be automatically
numbered)

Entering text
1)Use ctrl-c and ctrl-v to copy and paste marked text
passages into Toolbox
2)When the text is added one should press enter so
that \ft (free translation) reappears

Entering text
After entering the text press Alt-I (\mb, \ge & \ps will
reappear.

Enter text into Toolbox line by line

Interlinearize the text
Press Alt-I

Interlinearize the text
Press Alt-I
Morpheme
is not recognized!

Morphemes

Morphemes
●
One can just adjust morphemes by putting a space
in between in the text line and adding it to the
dictionary
– Later Toolbox will remember the morpheme and parse
it correctly
– It might not work first time though

Morphemes
●
●
One can just adjust morphemes by putting a space
in between in the text line and adding it to the
dictionary
– Later Toolbox will remember the morpheme and parse
it correctly
– It might not work first time though
One can just wait until Toolbox “learns” a
morpheme
– if it parses talk and later talking, it will remember these
and later automatically find the -ing (e.g.) ending
– For this also dictionary entries need to be made

Making an entry
●
●
●
●
Mark a word in the line \mb
Click on this word using the right mouse button
Click 'Insert'
Enter the lexical properties into the dictionary
field at the bottom of the screen

Making an entry

Making an entry

Entering new text
Move the cursor at the end of the free translation line and
press enter.

Entering new text
The new reference can be assigned a number by selecting 'tools' from
the menu and selecting 'renumber text'.

The result should look like this:

Wordlist, concordance and
dictionary
●
●
●
●
The dictionary is entered manually as shown
A wordlist can be produced using the menu 'tools'
and in this menu 'wordlist' (or by pressing alt-l)
A concordance can be produced can be produced
by using the menu 'tools' and in this menu
'concordance' (or by pressing ctrl-l)
A new text window can be added into a new text
file by choosing the menu 'database' and where
one will find 'new record'

Wordlist

Concordance

Exporting
●
It is possible to export the Toolbox data into a
word processor file

Exporting
●
It is possible to export the Toolbox data into a
word processor file
NB: Every active
window can be
exported!

The Dictionary in RTF
To get 'headers' for each letter use modify and tick the
appropriate box before exporting.

Adding new data categories
To add a new data category (e.g. the pronunciation)
click on the left column in the text window and
press ctrl-e.
Note on dictionary making:
What is seen in the left column of the text
window is called data categories (or
datcats), which are called fields in Toolbox,
what can be seen on the right side is
(language) data or records as Toolbox
refers to them.
The fields represent the microstructure of a
dictionary.

Saving
Do not forget to save the results. For this one has to
'file' under menu and click on 'SAVE ALL'!

References/Further Reading
For further reading use the reference/basis of this
presentation which is the Toolbox Manual/Guide
which is part of Toolbox Training which can be
downloaded on the SIL International website (also
from the 'Toolbox Home').
http://www.sil.org/computing/toolbox/downloads.htm
&
Toolbox website:
http://www.sil.org/computing/catalog/show_software.asp
15/12/05 01:11:17