Daniel Voigt Godoy - Deep Learning with PyTorch Step-by-Step A Beginner’s Guide-leanpub

watch1 = """
The Hatter was the first to break the silence. `What day of the
month is it?' he said, turning to Alice: he had taken his watch out
of his pocket, and was looking at it uneasily, shaking it every now
and then, and holding it to his ear.
"""
watch2 = """
Alice thought this a very curious thing, and she went nearer to
watch them, and just as she came up to them she heard one of them
say, `Look out now, Five! Don't go splashing paint over me like
that!
"""
sentences = [watch1, watch2]
In flair, every sentence is a Sentence object that’s easily created using the
corresponding text:
from flair.data import Sentence
flair_sentences = [Sentence(s) for s in sentences]
flair_sentences[0]
Output
Sentence: "The Hatter was the first to break the silence . ` What
day of the month is it ? ' he said , turning to Alice : he had taken
his watch out of his pocket , and was looking at it uneasily ,
shaking it every now and then , and holding it to his ear ." [
Tokens: 58]
Our first sentence has 58 tokens. We can use either the get_token() method or
the tokens attribute to retrieve a given token:
flair_sentences[0].get_token(32)
950 | Chapter 11: Down the Yellow Brick Rabbit Hole