About tree search About TGrep-lite About XPath
The tree search interfaces on this website present views of the parsed data generated from XML source files using the Alpino XML encoding (see below). The most direct way to search, as well as being the method of search with the most expressive power, is to enter XPath 2.0 queries directly into any of the text boxes in the interfaces. Knowing about the format in which the data is distributed is essential information for writing XPath queries. Before proceeding further on this page, we suggest that you find an example in the Pattern Browser, open a tree view of the example, then download the “XML tree”, and look at the names of the elements, the names of the attributes, and the values of the attributes. The rest of this section describes the XPath syntax used to query the data.
XPath can refer to the hierarchical information of Alpino XML as the embedding of “node” elements, grammatical categories and parts-of-speech by the “cat” attribute, word content by the “word” attribute, further word information by the “pt” attribute (such as whether the word is a null element), and surface order with attributes “begin” and “end”.
As an introductory example, the query:
identifies nodes for which the value of the “cat” attribute equals “pp”. The double slash (“//”) notation of the query indicates that this node can appear anywhere in a tree structure. Conditions for this node are given between square brackets and often refer to particular values of particular attributes. Conditions can be combined using the boolean operators “and”, “or” and “not”. For example, the previous query can be extended to require that the PP node has a sentence initial placement:
Brackets can be used to indicate the intended structure of the conditions. For example, the following query will match either PP or ADVP nodes that are not in sentence initial positions:
Conditions can also refer to what is outside the node itself. The following query, imposes restrictions from the daughter node of PP by finding all sentences in which a PP occurs with a complement that is not NP. This works by requiring that there is a “cat” attribute, but that this should not start with “np”.
It is possible to shift the selection of the query further down inside a node with single slash notation. For example, the following query will refer to a noun that is inside an NP-OB1:
XPath expressions can select nodes using a number of different axis specifiers. Each axis specifier describes a different set of nodes relative to the context node, where the context node is the central node in the following diagram:
The following is a description of the axis specifiers:
ancestor::
Refers to all ancestors of the context node: its parent, its parent's parent, and so on up to and including the root node.
ancestor-or-self::
Refers to the context node and its ancestors.
attribute::
Selects only the attributes of the context node. Can be abbreviated as “@”
child::
Selects children of the context node. Attributes are not included. The child axis is the default axis.
descendant::
Refers to the descendants of the context node: its children, their children, and so on down to and including the leaves of the tree. “descendant::*/” can be abbreviated as “.//”.
descendant-or-self::
Refers to the context node and its descendants.
following::
All nodes that follow the context node in the whole tree. This set does not include the context node's descendants or attributes.
following-sibling::
Refers to all children of the context node's parent that occur after the context node.
parent::
Selects only the parent node, if there is one. “parent::*” can be abbreviated as “..”
preceding::
All nodes that precede the context node in the whole tree. This set does not include the context node's descendants or attributes.
preceding-sibling::
Refers to all children of the context node's parent that occur before the context node.
self::
Selects only the context node. “self::*” can be abbreviated as “.”
To select nodes from a specific axis, use the syntax axis::e where e is any XPath expression. For example, the next query finds embedded clauses, so IP nodes that are not immediately under the root alpino_ds node, and that do not have a parent (e.g., a CP) that is immediately under the root alpino_ds node.
The next query finds embedded clauses that have the word ta somewhere as a descendant.
which can also be written as:
Typical attributes with numeric values are “begin”, “end” and “index”. The following query finds a contentful instance of -OB1 that preceeds an instance of -SBJ by requiring the “begin” numeric value of a node with -SBJ and not -SBJ2 to be before the “begin” numeric value of a sister node with -OB1 that does not contain a node with “zero” as a part-of-speech value.
Note how this query explicitly converts the value of the attributes “@begin” and “@end” to numeric values in order for the numeric comparison to succeed. The operators which require the conversion include “<”, “>” and “=”.
The above query can also be accomplished with the preceding-sibling:: axis descriptor:
To find occurrences of a bigram, e.g. “koozen to”, use:
This technique can be extended to longer N-grams, e.g. to find “o _ ga”, where “_” is any word, use:
The following query finds noun phrases that have more than one noun daughter, with count(node[@cat='n']) to count the number of nouns:
The above query selects the noun phrase. Selection can be left on the contained nouns with:
This section presents details for how parsed content is encoded in Alpino XML. Under this XML-format, first developed for treebanks of Dutch (van Noord et al 2013), nodes of tree structure are encoded by a recursive XML element “node”. Other information is presented as values of various XML-attributes of those nodes, including:
| cat | syntactic category or part-of-speech for the node |
| pt | further word information (such as whether the word is a null element) |
| word | realised word or null element |
| begin | starting position of the node with respect to the surface string |
| end | end position of the node with respect to the surface string |
As an example, consider:
<alpino_ds id="2_textbook_kisonihongo;page_13;JP" version="1.3">
<node cat="ip-mat" id="1" begin="0" end="5">
<node cat="np-sbj" sort="man_2" id="2" begin="0" end="1">
<node pt="zero" word="*pro*" id="3" begin="0" end="1">
</node>
</node>
<node cat="pp-ob1" id="4" begin="1" end="3">
<node cat="np" id="5" begin="1" end="2">
<node cat="n" id="6" begin="1" end="2">
<node pt="word" word="wa" id="7" begin="1" end="2">
</node>
</node>
</node>
<node cat="p-role" id="8" begin="2" end="3">
<node pt="word" word="o" id="9" begin="2" end="3">
</node>
</node>
</node>
<node cat="vb" id="10" begin="3" end="4">
<node pt="word" word="mawasu" id="11" begin="3" end="4">
</node>
</node>
<node cat="pu" id="12" begin="4" end="5">
<node pt="word" word="." id="13" begin="4" end="5">
</node>
</node>
</node>
<sentence>*pro* wa o mawasu .</sentence>
</alpino_ds>
This XML can be presented as a tree as follows:
The key innovation of this XML format is that the surface order of nodes is explicitly encoded by the XML attributes “begin” and “end” with each “node” element. This information allows querying the linear axis of the tree. For example, a query that requires a node x to immediately follow a node y can be encoded by requiring that the “begin”-attribute of x equals the “end”-attribute of y. This also enables querying constituent length, what resides at the left and right edges of given constituents, etc.
Non-terminal nodes have an attribute “cat” to represent the syntactic tag or part-of-speech tag. The attribute “cat” is the syntactic tag including any functional information for the node (e.g., “ip-mat” for matrix clause, “np-sbj” for subject noun phrase, “vb” for verb). Leaf nodes have an attribute “pt” to represent word information (e.g., “zero” for null elements).
The TGrep language by Richard Pito formulates queries as patterns that consist of expressions to match tree nodes and relationships defining links or negated links to other tree nodes. Nodes of searched trees are matched either with simple character strings or regular expressions (see sections 1). A complex expression consists of a node expression followed by relationships, as presented in section 2. Possible relationships are illustrated in section 3.
When you compose a search expression in the box provided, if the search expression uses TGrep-lite syntax, the interface will recognize that and interpret the expression accordingly. If the expression is well-formed but there are no matches in the corpus, the screen shows no change after the “Submit” button is pressed. If the expression is not well-formed, the warning “Not a valid TGrep-lite expression.” appears. In the result page, if you check the “reveal” box and resubmit, a translation of the TGrep-lite expression into XPath syntax is displayed. This allows you to check whether the expression you have composed reflects the search that is intended.
The TGrep functionality available here is referred to as “TGrep-lite”. This is less expressive than the full TGrep language implemented in the original TGrep program. In particular, the expression of relationships between nodes is limited to the relations detailed in section 3. TGrep-lite is especially weak when compared to enhanced TGrep languages available with TGrep2 and Tregex implementations, notably, missing the ability to express disjunctions of relations. TGrep-lite will also exhibit behaviour distinct from what is expected from TGrep with regards to how nodes are specified (described in section 1). Despite these mentioned limitations, TGrep-lite is the easiest and most accessible way to search the corpus using this on-line interface, and it is a powerful search language.
TGrep-lite works by rewriting expressions of a modified TGrep language into XPath queries over a database of XML encoded trees. The formatting of the XML requires that the rewrite to XPath distinguishes three different “node” kinds expressed with TGrep-lite node patterns:
The wild card (“__”) is exceptional in not needing to distinguish its node kind, since it will match all nodes. A simple constant string, such as “abc”, etc., will match word nodes that are the unique string abc. The expression of all other node patterns occurs as the statement of a regular expression with deliminators to determine the kind of the node searched. Specifically:
If a simple constant string or deliminated regular expression begins with “!”, the matching process will be complemented. That is, matches will turn into non-matches, and vice-versa. For example, “!abc” will match all words that are not abc, and “![^NP] will match any part-of-speech or phrase-level node that does not start with NP.
Specified as a string, a regular expression matches a node if there is a part of the node that is matched. For example, “[IP]” matches IP-MAT, IP-ADV, etc. The caret (“^”) anchors the regular expression to the beginning of a matched node, while a dollar sign (“$”) as the last character will anchor the regular expression to the end of a matched node. Use of both the caret and dollar-sign in “[^NP$]” constrains the match to only NP. A word boundary can be stated with “\b”. Thus, while “[^NP]” will match both NP-SBJ and NPR, “[^NP\b]” will match only NP-SBJ. Disjunction can be expressed with the pipe (“|”), and regular expression elements can be grouped with round brackets, such that “[^NP-(SBJ|OB1)]” will find nodes that start with either NP-SBJ or NP-OB1.
Note that the on-line interface is case insensitive when the node is identified as being either a pre-terminal or part-of-speech/phrase-level node, while being case sensitive for word (terminal) nodes.
TGrep-lite expressions are composed of a node pattern followed by the relationships the node pattern participates in. Because word information serves as content of the same node under the XML encoding as pre-terminal node information, it becomes necessary if you wish to match the combination of a particular word with a particular pre-terminal node that the “==” (equals) relation serves to connect this information about the same underlying node. For example, the following will find instances of words that contain “pro” with the “ZERO” pre-terminal tag.
The following example,
will match an IP node which immediately dominates a PP node and which dominates an IP node. Note the parenthesis to ensure that the second relationship “<< [IP]” refers to the first IP and not to the PP. As another example,
will match an IP which immediately dominates a PP which in turn dominates some IP.
The first node in a pattern or the first node following a left parenthesis is a “master” node which is related to the relationships to its right. Thus, a TGrep-lite pattern consists of a master node for the entire query followed by a series of relationships to other nodes that can themselves with parenthesis form master nodes with relationships to yet other nodes. In the first example above only the first [IP] is a master node, while in the second example both the first [IP] and the [PP] are master nodes.
Relationships define connections between the master node (being defined) and other nodes. There is a complete pairing of forward and backward links, allowing for flexibility in choosing what is the master node. Notable relationships are:
A << B A dominates (is an ancestor of) B A >> B A is dominated by (is a descendant of) B A < B A immediately dominates (is the parent of) B A > B A is immediately dominated by (is the child of) B A .. B A precedes B A ,, B A follows B A . B A immediately precedes B A , B A immediately follows B A $ B A is a sister of and not equal to B A $.. B A is a sister of and precedes B A $,, B A is a sister of and follows B A $. B A is a sister of and immediately precedes B A $, B A is a sister of and immediately follows B A $, B A is a sister of and immediately follows B A == B A and B are the same node A <<, B B is a leftmost descendant of A A <<- B B is a rightmost descendant of A A >>, B A is a leftmost descendant of B A >>- B A is a rightmost descendant of B A <1 B B is the 1st child of A A >1 B A is the 1st child of B A <-1 B B is the last child of A A >-1 B A is the last child of B A <, B B is the first child of A (synonymous with A <1 B) A >, B A is the first child of B (synonymous with A >1 B) A <- B B is the last child of A (also synonymous with A <-1 B) A >- B A is the last child of B (also synonymous with A >-1 B) A <: B B is the only child of A A >: B A is the only child of B A <<: B A dominates B via an unbroken chain (length > 0) of unary branches A >>: B A is dominated by B via an unbroken chain (length > 0) of unary branches
The following presents pictures grouping some of the above relationships as forward and backward links:
C << __ (dominates, is an ancestor of)
__ >> C (is dominated by, is a descendant of)
E >> __ (is dominated by, is a descendant of)
__ << E (dominates, is an ancestor of)
C > __ (immediately dominates, is the parent of)
__ < C (is immediately dominated by, is the child of)
I .. __ (precedes)
__ ,, I (follows)
I ,, __ (follows)
__ .. I (precedes)
I . __ (immediately precedes)
__ , I (immediately follows)
I , __ (immediately follows)
__ . I (immediately precedes)
F $ __ (sister)
__ $ F (sister)
E $.. __ (sister and precedes)
__ $,, E (sister and follows)
E $. __ (sister and immediately precedes)
__ $, E (sister and immediately follows)
An exclamation mark (!) can be placed immediately before any relationship to negate it. Thus, A !.. B means that A is not followed by B.
TGrep-lite returns the match for the left-most element in the search pattern. The following pattern matches PPs that are immediately dominated by an IP that dominates an IP:
Either a TGrep-lite expression, or an XPath query can be entered into the text area below.
For information on how to compose a TGrep-lite expression, click the above “About TGrep-lite” button.
For information on how to compose an XPath expression, click the above “About XPath” button. The XPath discussion includes information about the Alpino XML format that is used to encode the parsed annotation.
Clicking the “Submit” button triggers the search.
Note that search results can be filtered according to the source of the text.