view release on metacpan or  search on metacpan

share/doc/YaTeA/ArchitectureDescrYatea.tex  view on Meta::CPAN

extraction process implemented in \YaTeA.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

%% The term extractor aims at identifying and extracting noun phrases
%% which are potential terms ({\em i.e. } term candidates). Moreover each
%% term candidate is syntactically analysed in order to identify head and
%% modifier components. In that respect, it provides the syntactic head
%% of the term which is used in the syntactic parse step of the whole NLP
%% chain. Before parsing, terms identified in the corpus are simplified by their
%% syntactic head in order to decrease the complexity of the sentences.

%% We plan to set a confidence degree to each step of the term
%% analysis. 


\subsection{Endogenous and exogenous disambiguation}
\label{sec:endogenous}

Endogenous disambiguation consists in the exploitation of intermediate
chunking and parsing results for the parsing of a given Maximal Noun Phrase
(MNP). This feature
allows the parse of complex noun phrases using a limited number of
simple parsing patterns (80 patterns containing a maximum of 3
content words in the experiments described below). All the MNPs
corresponding to parsing patterns %  or testified terms
are
parsed first. In a second step, remaining unparsed MNPs are processed
using the results of the first step as \emph{islands of reliability}. 
An \emph{island of reliability} is a subsequence
(contiguous or not) of a MNP that corresponds %% This subsequence
to % a testified term or
a shorter term candidate that was
parsed during the first step of the parsing process. This subsequence along with its internal analysis is used as
an anchor in the parsing of the MNP. Islands are used to simplify the
POS sequence of the MNP for which no parsing pattern was found. The subsequence covered by the island is
reduced to its syntactic head. In addition,   
% As a consequence,
islands increase the degree of reliability of the parse as shown in Figure \ref{fig:endogenous}.  When no resource is provided and as there is no parsing pattern defined for the complete POS sequence "NN NN NN of NN" corresponding to the term candida...
\begin{figure}[!htbp]
\centering
  %% \textbf{differential pattern of cot gene expression}\\
%%   \textbf{JJ NN of NN NN NN}\\
%%   ISLANDS: \\
%%   - cot gene : head = gene\\
%%   - gene expression : head = expression\\
%%   REDUCED POS SEQUENCE:   JJ NN of NN\\
%%   PARSE using islands:  ( ( differential  pattern ) of  ( ( cot gene )
%%   expression ) ) \\
%%    PARSE without using islands: *( differential ( pattern of ( cot
  %%    ( gene  expression ) )\\
  % \includegraphics[scale=0.6]{endogenous}
%   \caption{Endogenous disambiguation for parsing} 
\includegraphics[scale=0.6]{exogenous}
  \caption{Effect of an isalnd of reliability}
  \label{fig:endogenous}
\end{figure}
%% The identification and the syntactic analysis of the terms is similar
%% to those used in the French term extractor Lexter \cite{lexter}:
%% identification of maximal noun phrases, analysis in head and modifer
%% of the noun phrase components. [J'AIME PAS TROP CETTE PHRASE]
%% The implementation of this term extractor allows to process large corpora.
%% The tool can be adapted to the corpus and a language thanks to the
%% configurable files defining the chunking and analysis of the input
%% data. In that respect, any part-of-speech tagset can be used.
%% We implement an endogenous  syntactic disambiguation of the terms
%% which uses  clues in the corpora to produce the best parse(s) for a term.
%% Exogeneous disambiguation is also made possible by using existing
%% terminologies (\textit{i.e.} testified terms) as an input of the extractor.
%% An index of reliability is assigned to each parse.
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

%\subsection{Exogenous disambiguation}
%\label{sec:exogenous}
\YaTeA allows exogenous dismabiguation,  \textit{i.e.} the exploitation of existing (attested) terminologies to
assist the chunking, parsing and extraction steps. 

During chunking, sequences of words corresponding to testified terms
are identified. They cannot be further split or deleted. Their POS tags and
lemmas can be corrected according to those associated to the testified term. 
If a MNP corresponds to a testified term for which a parse exists
(provided by the user or computed using parsing patterns), it is
recorded as a term candidate with the highest score of reliability.
Similarly to endogenous disambiguation, subsequences of MNPs
corresponding to testified terms are used to simplify the POS sequence
of the MNP and augment the quality of the parse%  (Figure \ref{fig:exogenous})
.
\subsection{Term candidate extraction process}\label{sec:term-extr-proc}
A noun phrase is extracted from the corpus and considered a term candidate if at least one
parse is found for it. This is performed in three main steps,
\emph{chunking}, \textit{i.e.} construction of a list of Maximal Noun
Phrases (MNPs) from the corpus, \emph{parsing}, \textit{i.e.} attempts
to find at least one syntactic parse for each MNP and, finally,
\emph{extraction} of term candidates.
The result of the term extraction process is two lists of noun phrases:  one
contains parsed MNPs, called \emph{term candidates}, the other
contains MNPs for which no parse was found. Both lists are proposed to
the user through a validation interface\footnote{ongoing development}.

%  We assume that the terms are
% part of the parsed maximal noun phrases and thus consider each
% internal node of the syntactic tree of a MNP as a
% potential term, i.e. a term candidate.

% Linguistic data defined by the user are used in the chunking and parsing steps as well as
% existing terminologies when provided.




% The term extractor has three steps: (1) the chuncking (identification of
% the maximal noun phrases), (2) the analysis of the term candidates fully
% covered by at least one parsing pattern (defined maximal noun phrases), and (3) the
% analysis of the remaining terms (under-defined  maximal noun phrases)
% using the results of step 2.

% It requires several types of information:
% \begin{itemize}
% \item chunking frontiers, used to identify the maximal noun phrases;

% \item cleaning frontiers, used to remove words (\emph{e.g.} determiners) that