Research Spotlight: Parsing Arabic Dialects
This summer, Prof. Rebecca Hwa, along with researchers from Columbia University, the University of Maryland and the University of Amsterdam, is participating in an NSF-sponsored workshop on Parsing Arabic Dialects. Students from Stanford University, The University of Pittsburgh, Johns Hopkins University and Georgia Tech are also participating in the project.
The project will tackle the problem of parsing Arabic dialects. Parsing is an important component in many advanced NLP systems, and has also proven useful for language modeling for ASR. As is well known, Arabic exhibits diglossia, i.e., the coexistence of two forms of language, a high variety with standard orthography and sociopolitical clout which is not natively spoken by anyone (Modern Standard Arabic, MSA) and low varieties that are primarily spoken and lack writing standards (Arabic dialects). The dialects and MSA form a continuum of variation at the lexical, phonological, morphological, and syntactic levels.
There are important resources currently available for MSA with much ongoing NLP work; for example, there are several syntactic and semantic parsers for MSA. However, Arabic dialect resources and NLP research are still at an infancy stage. There are linguistic studies of Arabic dialectal syntax but there is no language engineering work (such as computational grammars). There are no parallel written corpora between any of the dialects and any other language, including MSA. Thus, most of the techniques developed for parsing that exploit supervised (in the canonical sense) machine learning do not apply, since there is no sufficient annotated data to learn from. The researchers would like to leverage existing resources and tools for MSA in order to parse Arabic dialects using both symbolic techniques and machine learning approaches.
The investigators expect that the project will have a significant impact on many areas:
- General NLP research: researchers will investigate how to leverage available syntactic resources for families of resource-poor languages.
- Tools: researchers will create standard tools, i.e. parsers with compatible tokenization and morphological analysis components, for the processing of Arabic (MSA and dialects). These can be used in applications such as dialect translation, information retrieval, information extraction from speech data, dialect transcription, language modeling for ASR, and semantic parsing of Arabic dialects.
- Resources: researchers will create standards for the transcription of Arabic dialects, as well as grammars and small corpora and lexica.