Extraction and Integration of MovieLens and IMDb Data - APMD

More documents

Recommendations

Info

Extraction and Integration of MovieLens and IMDb Data – Technical Report Tagged files precedes data with tags, which indicates the nature of data. Figure 18 shows an extract of the biographies.list file; tags are placed at the start of each line, indicating for example, an artist name (NM), his real name (RN), his date and place of birth (DB), etc. Tags are prefixed for each list but some tags are not explained. This type of file needs ad-hoc parsing. The files having this format are: 14 − biographies.list includes as tags: artist name, real name, birth (date and place), decease (date, place and cause) and height. Other tags are ignored. − business.list includes as tags: movie title, budget (currency and amount), and revenue (currency, amount and country). Other tags are ignored. NM: Noth, Chris RN: Noth, Christopher David DB: 13 November 1954, Madison, Wisconsin, USA BG: Chris Noth was born in Madison, Wisconsin on 13 November 1954. In his BG: youth he travelled and lived in England, Yugoslavia and Spain… Figure 18 – Extract of the biographies.list file Hierarchical-structured files organize data into two hierarchical levels: the outside level describes a movie feature (e.g. an actor or a director) and the inside level lists all movies corresponding to such feature. Figure 19 shows an extract of the actors.list file, presenting the list of movies played by an actor. Some files contains other attributes and comments, which follow movie titles (in the same line) separated by one or more tabulations. This type of file is the most difficult to extract; it needs ad-hoc extractors to iterate between hierarchical levels, and pre-processing for solving tabulations problems (see tab-separated columned files). The files having this format are: − actors.list lists actors and 4 columns (movie title, role, casting position, comments). Columns (excepting movie title) are optional. − actresses.list lists actresses and 4 columns (movie title, role, casting position, comments). Columns (excepting movie title) are optional. − costume-designers.list lists costume designers and 2 columns (movie title, comments). − directors.list lists directors and 2 columns (movie title, comments). − distributors.list lists movie titles and 2 columns (distribution company, comments), where distribution company may contain acronyms (between brackets) and the code of one or several countries (between square brackets), for example: “France 2 (FR 2) [fr]” − movie-links.list lists movie titles and 1 columns (link to another movie title), which embeds the link type (e.g. “featured in”, “alternate language version of”, “referenced in”, etc). − producers.list lists producers and 2 columns (movie title, comments). − production-designers.list lists producion designers and 2 columns (movie title, comments). − writers.list lists custome designers and 2 columns (movie title, comments). Outstanding Supporting Actor in a Comedy Series] Noth, Chris Abducted: A Father's Love (1996) (TV) [Larry Coster] Acting Class, The (2000) [Martin Ballsac] Apology (1986) (TV) [Roy Burnette] At Mother's Request (1987) (TV) [Steve Klein] Baby Boom (1987) (as Christopher Noth) [Yuppie Husband] Bad Apple (2004) (TV) [Tozzi] Figure 19 – Extract of the actors.list file Table 5 summarizes file formats, optional attributes and other particularities.
Verónika Peralta File Format Optional att. Particularities movies.list Tab-separated columned actors.list Hierarchical-structured comments, role, casting actresses.list Hierarchical-structured comments, role, casting biographies.list Tagged Columns birth and decease are commaseparated lists of dates and places. business.list Tagged Columns budget and revenue embed currencies, amounts and countries. color-info.list Tab-separated columned comments costume-designers.list Hierarchical-structured comments countries.list Tab-separated columned directors.list Hierarchical-structured comments distributors.list Hierarchical-structured comments Column distribution company embeds company acronyms and country code genres.list Tab-separated columned keywords.list Tab-separated columned Contains two lists: (i) list of keywords, (ii) lists of movies and their keywords. language.list Tab-separated columned comments locations.list Tab-separated columned comments Column location is a comma-separated list of geographical places. movie-links.list Hierarchical-structured Column link embeds link type. producers.list Hierarchical-structured comments production-companies.list Tab-separated columned comments Column production company embeds country code production-designers.list Hierarchical-structured comments ratings.list Fixed-length columned release-dates.list Tab-separated columned comments Column release date embeds country. running-times.list Tab-separated columned comments Column running time embeds country. sound-mix.list Tab-separated columned writers.list Hierarchical-structured comments Table 5 – List formats and particularities Movies correspond to one of 6 types: cinema film, series episode, mini-series, TV series, video and video game. Movie identifiers (MovieTitle columns) are Strings that concatenates different data depending on movie types: − Cinema film identifiers concatenate movie title and release year (between brackets). E.g.: Walk in the Clouds, A (1995) − Series episode identifiers concatenate movie title (quoted), release year (between brackets) and episode identification (between braces). The latter may include episode name, episode number (between brackets), or both. E.g.: "Sex and the City" (1998) {All or Nothing (#3.10)} − Mini-series identifiers concatenate movie title (quoted), release year (between brackets) and the word mini (between brackets). E.g.: "Bon Voyage" (2006) (mini) − TV series identifiers concatenate movie title, release year (between brackets) and the word TV (between brackets). E.g.: Aladdin on Ice (1995) (TV) − Video identifiers concatenate movie title, release year (between brackets) and the letter V (between brackets). E.g.: Elton John: Live in Barcelona (1992) (V) − Video game identifiers concatenate movie title, release year (between brackets) and the word VG (between brackets). E.g.: Harry Potter: Quidditch World Cup (2003) (VG) 3.2. IMDb target schema The target schema for the IMDb data set consists in 24 tables describing movies and companies or persons related to movies. Figure 20 shows the tables of the target schema, their primary keys (underlined attributes) and foreign keys (arrows between tables). Additional dotted lines indicate is-a relationships among some tables and a (not implemented) relation describing persons. Table 6 describes each target table, its attributes and constraints. 15
Page 1 and 2: Extraction and Integration of Movie
Page 3 and 4: Movies.dat 1::1193::5::978300760 1:
Page 5 and 6: u.genre Verónika Peralta This file
Page 7 and 8: Table Attributes Constraints SML_Us
Page 9 and 10: Verónika Peralta temporal table st
Page 11 and 12: Verónika Peralta − Manual insert
Page 13: Verónika Peralta corrections and a
Page 17 and 18: Table Attributes Constraints IMDb_M
Page 19 and 20: IMDb_ReleaseDates Dates of movie re
Page 21 and 22: Verónika Peralta In order to store
Page 23 and 24: Verónika Peralta − Movie type ca
Page 25 and 26: Verónika Peralta The implementatio
Page 27 and 28: ML_Movies SML_Movies IMDb_Movies ma
Page 29 and 30: 4.2. Integrated schema Verónika Pe
Page 31 and 32: I_MovieCountries Countries of movie
Page 33 and 34: I_Users Descriptive information abo
Page 35 and 36: Verónika Peralta integration proce

Extraction and Integration of MovieLens and IMDb Data - APMD

Create successful ePaper yourself

Delete template?

Save as template?