<![CDATA[☆☆ 卢伟个人主页 _____ ☆☆☆☆☆ _____ Lu Wei's Homepage ☆☆]]>http://www.luweixmu.com/home/PJBlog32012-03-20T00:24:36+08:00<![CDATA[New Publications of Linguistic Data Consortium]]>adminhttp://www.luweixmu.com/home/luwig@xmu.edu.cn2012-03-20T00:24:36+08:002012-03-20T00:24:36+08:00 - 2012 LDC Survey Responses and Benefit Winner - - LDC at ICASSP 2012 -
New publications:
LDC2012T02- English Translation Treebank: An Nahar Newswire - LDC2012S04 - Malto Speech and Transcripts - -------------------------------------------------------------------------------- 2012 LDC Survey Responses and Benefit Winner
Thanks to all who participated in the 2012 LDC Survey. Your responses were thoughtful and informative. We’re now analyzing the results; stay tuned for an announcement on the survey findings.
In the meantime, please join us in congratulating Todor Ganchev from the University of Patras, Wire Communications Laboratory (WCL) for winning the survey participation benefit! As a reminder, one $500 benefit was awarded to a blindly-selected participant whose response was received by February 7, 2012.
LDC at ICASSP 2012
LDC will be traveling across the globe to exhibit at its first IEEE-hosted event. The 37th International Conference on Acoustics, Speech, and Signal Processing (ICASSP) will be held at the Kyoto International Conference Center in Kyoto, Japan, on March 25 - 30, 2012.
The ICASSP meeting is the world’s largest and most comprehensive technical conference focused on signal processing and its applications, and LDC is looking forward to interacting with members of this community. Please look for LDC’s exhibition at Booth #14 in the Annex Hall. We hope to see you there!
New Publications
(1) English Translation Treebank: An Nahar Newswire was developed by LDC and consists of 599 distinct newswire stories from the Lebanese publication An Nahar translated from Arabic to English and annotated for part-of-speech and syntactic structure.
This corpus is part of an ongoing effort at LDC to produce parallel Arabic and English treebanks. The guidelines followed for both part-of-speech and syntactic annotation are Penn Treebank II style, with changes in the tokenization of hyphenated words, part-of-speech and tree changes necessitated by those tokenization changes and revisions to the syntactic annotation to comply with the updated annotation guidelines (including the "Treebank-PropBank merge" or "Treebank IIa" and "treebank c" changes). The original Penn Treebank II guidelines, addenda describing changes to the guidelines and the tokenization specifications can be found on LDC's website.
The data consists of 461,489 tokens in 599 individual files. The news stories in this release were published in An Nahar in 2002.
The English sources files (translated from the Arabic) were automatically tokenized, part-of-speech tagged and parsed; the tokens, tags and parses were manually corrected. The quality control process consisted of a series of specific searches for over 100 types of potential inconsistency and parse or annotation error. Any errors found in those searches were manually corrected.
Annotations are in the following two formats:
Penn Style Trees Bracketed tree files following the basic form (NODE (TAG token)). Each sentence is surrounded by a pair of empty parentheses. AG xml TreeEditor .xml stand-off annotation files. These files contain the POS and Treebank annotation and reference the source files by character offset. DTD files for the AG xml files were moved from their original location indicated in the readme to be more consistent with LDC publications. English Translation Treebank: An Nahar Newswire is distributed via web download.
2012 Subscription Members will automatically receive two copies of this corpus on disc. 2012 Standard Members may request a copy as part of their 16 free membership corpora. Non-members may license this data for US$4500.
(2) Malto Speech and Transcripts was developed by Masato Kobayashi, Associate Professor in Linguistics at the University of Tokyo (Japan), and Bablu Tirkey, research scholar at the Tribal and Regional Languages Department, Ranchi University (India). It contains approximately 8 hours of Malto speech data collected between 2005 and 2009 from 27 speakers (22 males, 5 females). Also included are accompanying transcripts, English translations and glosses for 6 hours of the collection. Speakers were asked to talk about themselves, their lives, rituals and folklore; elicitation interviews were then conducted. The goal of the work was to present the current state and dialectal variation of Malto.
Malto is a Dravidian language spoken in northeastern India (principally the states of Bihar, Jharkhand and West Bengal) and Bangladesh by people called the Pahariyas. Indian census data places the number of Malto speakers in a range of between 100,000-200,000 total speakers. Most Malto speakers live in the three northeastern districts of Jharkhand, i.e, Sahebganj, Godda and Pakur; the fieldwork that resulted in this corpus was conducted in those districts. Of the Pahariyas in that area, three subtribes, the Sawriya Pahariyas, the Mal Pahariyas and the Kumarbhag Pahariyas, primarily speak Malto.
The transcribed data accounts for 6 hours of the collection and contains 21 speakers (17 male, 4 female). The untranscribed data accounts for 2 hours of the collection and contains 10 speakers (9 male, 1 female). Four of the male speakers are present in both groups.
All audio is presented in .wav format. Each audio file name includes a subject number, village name, speaker name and the topic discussed. The transcripts and glossary are UTF-8 text files. Because of ambiguities that occur when writing Malto in Devenagari script, the transcripts were developed using Roman script with symbols adapted from the International Phonetic Alphabet (IPA) but are not considered phonetic transcripts.
Malto Speech and Transcripts is distributed on 1 DVD-ROM.
2012 Subscription Members will automatically receive two copies of this corpus. 2012 Standard Members may request a copy as part of their 16 free membership corpora. The first 100 copies distributed to non-member organizations are available at no charge. Shipping and handling fees apply. -------------------------------------------------------------------------------- Ilya Ahtaridis Membership Coordinator -------------------------------------------------------------------- Linguistic Data Consortium Phone: 1 (215) 573-1275 University of Pennsylvania Fax: 1 (215) 573-2175 3600 Market St., Suite 810 ldc@ldc.upenn.edu Philadelphia, PA 19104 USA http://www.ldc.upenn.edu]]>http://www.luweixmu.com/home/default.asp?id=184<![CDATA[New Publications of Linguistic Data Consortium]]>adminhttp://www.luweixmu.com/home/luwig@xmu.edu.cn2012-03-20T00:23:08+08:002012-03-20T00:23:08+08:00 - Spring 2012 LDC Data Scholarship Recipients! -
- Membership Fee Savings and Publications Pipeline for MY2012 -
New publications:
LDC2012S03 - Digital Archive of Southern Speech (DASS) -
LDC2012T01 - ModeS TimeBank 1.0 - -------------------------------------------------------------------------------- Spring 2012 LDC Data Scholarship Recipients!
LDC is pleased to announce the student recipients of the Spring 2012 LDC Data Scholarship program! This program provides university students with access to LDC data at no-cost. Students were asked to complete an application which consisted of a proposal describing their intended use of the data, as well as a letter of support from their thesis adviser. We received many solid applications and have chosen six proposals to support. The following students will receive no-cost copies of LDC data:
Zainab Ali Khalaf – University of Science, Malaysia (Malaysia), graduate student, Computer Science. Zainab has been awarded a copy of 1996 English Broadcast News Transcripts (HUB4) (LDC97T22) for her work in spoken document retrieval.
Daniel Jettka – Trinity College Dublin (Ireland), graduate student, Centre for Language & Communication Studies. Daniel has been awarded copies of Penn Discourse Treebank Version 2.0 (LDC2008T05) and RST Discourse Treebank (LDC2002T07) for his work in anaphora resolution.
Olga Nickolaevna Ladoshko - National Technical University of Ukraine “KPI” (Ukraine), graduate student, Acoustics and Acoustoelectronics. Olga has been awarded copies of NTIMT (LDC93S2) and STC-TIMIT 1.0 (LDC2008S03) for her research in automatic speech recognition for Ukrainian.
Ming Yang, Xiaoxiao Ma, and Jiajia Huang – Wuhan University (China), graduate students, Computer Science. Ming, Xiaoxiao, and Jiajia have been awarded copies of ACE Time Normalization (TERN) 2004 English Training Data v 1.0 (LDC2005T07) and GALE Phase 1 Chinese Broadcast News Parallel Text – Part 1 (LDC2007T23) for their work in summarization and data mining.
Daria Vazhenina – University of Aizu (Japan), graduate student, Human Interface Lab. Daria has been awarded a copy of 2005 Spring NIST Rich Transcription (RT-05S) Evaluation Set (LDC2011S06) for her work in speaker diarization.
Tanina Zappone - University of Rome “La Sapienza” (Italy), graduate student, oriental Studies. Tanina has been awarded a copy of Chinese Treebank 7.0 (LDC2010T07) for her work in China’s political communications.
Please join us in congratulating our student recipients! The next LDC Data Scholarship program is scheduled for the Fall 2012 semester.
Membership Fee Savings and Publications Pipeline for MY2012
Time is quickly running out to save on membership fees for MY2012! Any organization which joins or renews membership for 2012 through Thursday, March 1, 2012, is entitled to a 5% discount on membership fees. organizations which held membership for MY2011 can receive a 10% discount on fees provided they renew prior to March 1, 2012.
Many publications for MY2012 are still in development. The planned publications for the upcoming months include:
ARRAU (Anaphor Resolution and Underspecification) ~ data annotated for anaphoric relations, with information about agreement and explicit representation of multiple antecedents for ambiguous anaphoric expressions and discourse antecedents for expressions which refer to abstract entities such as events, actions and plans. The corpus contains texts from various genres: task-oriented dialogues from the TRAINS project, narratives from the English Pear Stories , and newspaper articles from the Wall Street Journal portion of the Penn Treebank.
MALACH English ~ over 300 hours of English audio recordings of interviews conducted under the auspices of the USC Shoah Foundation Institute for Visual History and Education and associated transcripts produced as part of the Multilingual Access to Large Spoken ArCHives (MALACH) project.
Malto Speech and Transcripts ~ speech files of Malto narratives recorded by Masato Kobayashi and Bablu Tirkey with associated transcripts. Malto is a Dravidian language spoken in northeastern India and Bangladesh.
NIST/USF Evaluation Resources for the VACE Program – Broadcast News ~ English broadcast news video annotated for the VACE (Video Analysis and Content Extraction) 2005 face, text and text word detection and tracking tasks.
OntoNotes 5.0 ~ multiple genres of English, Chinese, and Arabic text annotated for syntax, predicate argument structure and shallow semantics.
2012 Subscription Members are automatically sent all MY2012 data as it is released. 2012 Standard Members are entitled to request 16 corpora for free from MY2012. Non-members may license most data for research use.
New publications
(1) Digital Archive of Southern Speech (DASS) was developed by the University of Georgia. It is a subset of the Linguistic Atlas of the Gulf States (LAGS), which is in turn part of the Linguist Atlas Project (LAP). DASS contains approximately 370 hours of English speech data from 30 female speakers and 34 male speakers in .wav format and in .mp3 format, along with associated metadata about the speakers and the recordings and maps in .jpeg format relating to the recording locations.
LAP consists of a set of survey research projects about the words and pronunciation of everyday American English, the largest project of its kind in the United States. Interviews with thousands of native speakers across the country have been carried out since 1929. LAGS surveyed the everyday speech of Georgia, Tennessee, Florida, Alabama, Mississippi, Arkansas, Louisiana, and Texas in a series of 914 audio-taped interviews conducted from 1968-1983. Interviews average approximately six hours in length; the systematic LAGS tape archive amounts to 5500 hours of sound recordings. DASS is a collection of 64 interviews from LAGS selected to cover a range of speech across the region and to represent multiple education levels and ethnic backgrounds.
Also included in this release is a version of the LICHEN software developed at the University of Oulu, Finland. LICHEN allows users to browse and search through the audio data in a more advanced fashion using a graphical interface.
Digital Archive of Southern Speech (DASS) is distributed on one hard disc drive.
2012 Subscription Not-for-Profit/US Government Members will automatically receive one copy of this data. 2012 For-Profit Members will receive a copy provided that they have submitted a completed copy of the User License Agreement for Digital Archive of Southern Speech (LDC2012S03). 2012 Standard Members may request a copy as part of their 16 free membership corpora. Non-members may license this data for US$250.
(2) ModeS TimeBank 1.0 was developed by researchers at Technical University of Madrid and Barcelona Media and is a corpus of Modern Spanish (17th and 18th centuries) annotated with temporal and event information according to TimeML mark-up and annotated with spatial information following the SpatialML scheme.
TimeML (Pustejovsky et al., 2005) is a specification language for annotating eventualities and time expressions in natural language as well as the temporal relations among them, thus facilitating the task of extraction, representation and exchange of temporal information. SpatialML (Mani et al., 2008) is a specification language for annotating and normalizing spatial expressions by means of geographic coordinates.
ModeS TimeBank 1.0 contains 102 documents reporting a sea-crossing cruise by a ship called La Princesa, which took place from December 1768 to April 1769. There exist copious logbooks from that period that not only provide information about shipping routes, but also contain valuable data concerning information flows, commercial agents and social networks.
All text is encoded in UTF-8. The data in ModeS TimeBank 1.0 has been tokenized, POS-tagged, and annotated with space, time and event information according to the TimeML and SpatialML specification schemes.
ModeS TimeBank 1.0 is distributed via web download.
2012 Subscription Members will automatically receive two copies of this corpus on disc. 2012 Standard Members may request a copy as part of their 16 free membership corpora. Non-members may request this data by completing a copy of the LDC User Agreement for Non-Members. The agreement can be faxed +1 215 573 2175 or scanned and emailed to this address. This data is available at no charge. -------------------------------------------------------------------------------- Ilya Ahtaridis Membership Coordinator -------------------------------------------------------------------- Linguistic Data Consortium Phone: 1 (215) 573-1275 University of Pennsylvania Fax: 1 (215) 573-2175 3600 Market St., Suite 810 ldc@ldc.upenn.edu Philadelphia, PA 19104 USA http://www.ldc.upenn.edu]]>http://www.luweixmu.com/home/default.asp?id=183<![CDATA[New Publications of Linguistic Data Consortium]]>adminhttp://www.luweixmu.com/home/luwig@xmu.edu.cn2012-03-20T00:21:46+08:002012-03-20T00:21:46+08:00 - LDC Celebrates its 20th Anniversary! - - 2012 LDC Survey – Be on the Lookout! - - Membership Discounts for MY 2012 Still Available -
New publications:
LDC2012S01 - 2006 NIST Speaker Recognition Evaluation Test Set Part 2 -
LDC2012S02 - TORGO Database of Dysarthric Articulation - -------------------------------------------------------------------------------- LDC Celebrates its 20th Anniversary!
2012 marks LDC’s 20th Anniversary year – officially on April 15 – but this is cause for a yearlong celebration! From our founding in 1992 as a data repository and language resource distribution center, our online catalog has grown to include over 500 databases in 60 languages that have been licensed by over 3000 organizations from 80 different nations. This data has been made available through donations, funded projects at LDC or elsewhere, community initiatives, and from LDC resources, an indication of the collective strength of this consortium. And, LDC has evolved from an organization that shares language resources to one that also is at the forefront of language technology research that includes the development of new data resources, software tools, and standards and best practices.
As we celebrate throughout the year, look for announcements and special features in our newsletter and on our Facebook page.
2012 LDC Survey – Be on the Lookout!
It’s been four years since our last survey of LDC members and data licensees and we would like to again ask you to share your views on LDC and its language resources as well as your thoughts about data distribution in general and the impact of social media on language-related research and technology development. These topics are particularly timely as LDC enters its 20th anniversary year.
The 2012 LDC Survey will be sent to every person and organization that licensed LDC data and/or joined LDC as a Member during the period from 2009 through 2011. Those who complete the survey on or before February 7, 2012 will make their organization eligible for a $500 benefit to be applied to any corpus or membership purchase in 2012. LDC will conduct a blind drawing and one lucky winner will be chosen from the pool of respondents.
Many thanks for your continued support and for your participation in the 2012 Survey!
Membership Discounts for MY 2012 Still Available If you are considering joining for Membership Year 2012 (MY2012), there is still time to save on membership fees. Any organization which joins or renews membership for 2012 through Thursday, March 1, 2012, is entitled to a 5% discount on membership fees. organizations which held membership for MY2011 can receive a 10% discount on fees provided they renew prior to March 1, 2012. For further information on pricing, please consult our Announcements page or contact LDC.
New Publications (1) 2006 NIST Speaker Recognition Evaluation Test Set Part 2 was developed by LDC and National Institute of Standards and Technology (NIST). It contains 568 hours of conversational telephone and microphone speech in English, Arabic, Bengali, Chinese, Farsi, Hindi, Korean, Russian, Spanish, Thai and Urdu and associated English transcripts used as test data in the NIST-sponsored 2006 Speaker Recognition Evaluation (SRE).
The task of the 2006 SRE evaluation was speaker detection, that is, to determine whether a specified speaker is speaking during a given segment of conversational telephone speech. The task was divided into 15 distinct and separate tests involving one of five training conditions and one of four test conditions. Further information about the test conditions and additional documentation is available at the NIST web site for the 2006 SRE and within the 2006 SRE Evaluation Plan.
LDC has previously published 2006 NIST Speaker Recognition Evaluation Training Set and 2006 NIST Speaker Recognition Evaluation Test Set Part 1.
The speech data in this release was collected by LDC as part of the Mixer project, in particular Mixer Phases 1, 2 and 3. The Mixer project supports the development of robust speaker recognition technology by providing carefully collected and audited speech from a large pool of speakers recorded simultaneously across numerous microphones and in different communicative situations and/or in multiple languages. The data is mostly English speech, but includes some speech in Arabic, Bengali, Chinese, Farsi, Hindi, Korean, Russian, Spanish, Thai and Urdu.
The telephone speech segments are multi-channel data collected simultaneously from a number of auxiliary microphones. The files are organized into four types: two-channel excerpts of approximately 10 seconds, two-channel conversations of approximately 5 minutes, summed-channel conversations also of approximately 5 minutes and a two-channel conversation with the usual telephone speech replaced by auxiliary microphone data in the putative target speaker channel. The auxiliary microphone conversations are also of approximately five minutes in length. English language transcripts in .ctm format were produced using an automatic speech recognition (ASR) system.
2006 NIST Speaker Recognition Evaluation Test Set Part 2 is distributed on seven DVD-ROM.
2012 Subscription Members will automatically receive two copies of this corpus. 2012 Standard Members may request a copy as part of their 16 free membership corpora. Non-members may license this data for US$2000.
(2) TORGO Database of Dysarthric Articulation was developed by the University of Toronto's departments of Computer Science and Speech Language Pathology in collaboration with the Holland-Bloorview Kids Rehabilitation Hospital in Toronto, Canada. It contains approximately 23 hours of English speech data, accompanying transcripts and documentation from 8 speakers (5 males, 3 females) with cerebral palsy (CP) or amyotrophic lateral sclerosis (ALS) and from 7 speakers (4 males, 3 females) from a non-dysarthric control group.
CP and ALS are examples of dysarthria which is caused by disruptions in the neuro-motor interface that distort motor commands to the vocal articulators, resulting in atypical and relatively unintelligible speech in most cases. The TORGO database is primarily a resource for developing advanced automatic speaker recognition (ASR) models suited to the needs of people with dysarthria, but it is also applicable to non-dysarthric speech. The inability of modern ASR to effectively understand dysarthric speech is a problem since the more general physical disabilities often associated with the condition can make other forms of computer input, such as computer keyboards or touch screens, difficult to use.
The data consists of aligned acoustics and measured 3D articulatory features from the speakers carried out using the 3D AG500 electro-magnetic articulograph (EMA) system (Carstens Medizinelektronik GmbH, Lenglern, Germany) with fully-automated calibration. This system allows for 3D recordings of articulatory movements inside and outside the vocal tract, thus providing a detailed window on the nature and direction of speech-related activity.
All subjects read text consisting of non-words, short words and restricted sentences from a 19-inch LCD screen. The restricted sentences included 162 sentences from the sentence intelligibility section of Assessment of intelligibility of dysarthric speech (Yorkston & Beukelman, 1981) and 460 sentences derived from the TIMIT database. The unrestricted sentences were elicited by asking participants to spontaneously describe 30 images in interesting situations taken randomly from Webber Photo Cards - Story Starters (Webber, 2005), designed to prompt students to tell or write a story.
Data is organized by speaker and by the session in which each speaker recorded data. Each speaker's directory contains 'Session' directories which encapsulate data recorded in the respective visit and occasionally, a 'Notes' directory which can include Frenchay assessments (test for the measurement, description and diagnosis of dysarthria), notes about sessions (e.g., sensor errors), and other relevant notes.
TORGO Database of Dysarthric Articulation is distributed on 4 DVD-ROM.
2012 Subscription Members will automatically receive two copies of this corpus. 2012 Standard Members may request a copy as part of their 16 free membership corpora. Non-members may license this data for US$1200. -------------------------------------------------------------------------------- Ilya Ahtaridis Membership Coordinator -------------------------------------------------------------------- Linguistic Data Consortium Phone: 1 (215) 573-1275 University of Pennsylvania Fax: 1 (215) 573-2175 3600 Market St., Suite 810 ldc@ldc.upenn.edu Philadelphia, PA 19104 USA http://www.ldc.upenn.edu]]>http://www.luweixmu.com/home/default.asp?id=182<![CDATA[New Publications of Linguistic Data Consortium]]>adminhttp://www.luweixmu.com/home/luwig@xmu.edu.cn2012-03-20T00:19:34+08:002012-03-20T00:19:34+08:00- Spring 2012 LDC Data Scholarship Program - deadline approaching! - - LDC Exhibiting at LSA 2012 Annual Meeting - - LDC Hosts Satellite Workshop at LSA 2012 - - LDC to Close for Winter Break -
New publications
LDC2011S10 - 2006 NIST Speaker Recognition Evaluation Test Set Part 1 - LDC2011S11 - 2008 NIST Speaker Recognition Evaluation Supplemental Set - --------------------------------------------------------------------------------
Spring 2012 LDC Data Scholarship Program - deadline fast approaching!
The deadline for the Spring 2012 LDC Data Scholarship Program is less than a month away! Applications are being accepted through January 15, 2012. The LDC Data Scholarship program provides university students with access to LDC data at no cost. This program is open to students pursuing both undergraduate and graduate studies in an accredited college or university. LDC Data Scholarships are not restricted to any particular field of study; however, students must demonstrate a well-developed research agenda and a bona fide inability to pay.
Students will need to complete an application which consists of a data use proposal and letter of support from their adviser. For further information on application materials and program rules, please visit the LDC Data Scholarship page.
Students can email their applications to the LDC Data Scholarship program. Decisions will be sent by email from the same address.
LDC Exhibiting at LSA 2012 Annual Meeting
LDC looks forward to mingling with linguists and language specialists when we exhibit at the 86th Annual Meeting of the Linguistic Society of America (LSA). The main conference will be held over January 5-8, 2012 at the Portland, or Hilton and Executive Tower and the exhibit hall will be open from January 6-8th (limited hours on Sunday the 8th). Please stop by our display for news on what 2012 will hold for LDC and to receive some of our conference giveaways.
LSA 2012 will feature plenary talks on the following topics:
Patrice Speeter Beddor (University of Michigan): "The Dynamics of Speech Perception: Constancy, Variation, and Change" Dan Jurafsky (Stanford University): "Computing Meaning: Learning and Extracting Meaning from Text" Ted Supalla (University of Rochester): "Rethinking the Emergence of Grammatical Structure in Signed Languages: New Evidence from Variation and Historical Change in American Sign Language" For further information visit the LSA Annual Meeting website. If you would like to learn more about LDC’s conference preparations, please ‘like’ our Facebook page. We hope to see you there!
LDC Hosts Satellite Workshop at LSA 2012
LDC will co-host a satellite workshop entitled Sociolinguistic Archival Preparation on January 4-5, 2012 in conjunction with the LSA 2012 Annual Meeting in Portland, or. This two-day workshop will focus on techniques to permit the archiving of data, for cross-community sharing of corpora as well as for subsequent 'panel' studies. Recent discussions within the field have concluded that present protocols need to be expanded to permit adequate archiving. Specifically:
Institutional Review Board (IRB) paperwork needs to be adapted to provide protection for interviewees while permitting their speech data to be more generally sharable (and therefore archiveable); Demographic, situational, and attitudinal protocols are needed to provide a unified resource serving multiple research communities as well as the contributing researchers. The sooner IRB forms and research protocols are aligned with each other, the sooner sharable, archiveable corpora will become available, permitting intergroup comparison and interdisciplinary collaboration.
LDC's Executive Director, Christopher Cieri, and LDC consultant and University of Arizona scholar, Malcah Yaeger-Dror, are the workshop organizers. This workshop is funded in part by the National Science Foundation (BCS#1144480). Further information about the workshop is available on the LSA Annual Meeting website.
LDC to Close for Winter Break
LDC will be closed from Monday, December 26, 2011 through Monday, January 2, 2012 in accordance with the University of Pennsylvania Winter Break Policy. Our offices will reopen on Tuesday, January 3, 2012. Requests received for membership renewals and corpora during the Winter Break will be processed at that time.
Best wishes for a happy and safe holiday season!
New Publications
(1) 2006 NIST Speaker Recognition Evaluation Test Set Part 1 was developed by LDC and National Institute of Standards and Technology (NIST). It contains 437 hours of conversational telephone and microphone speech in English, Arabic, Bengali, Chinese, Farsi, Hindi, Korean, Russian, Spanish, Thai and Urdu and associated English transcripts used as test data in the NIST-sponsored 2006 Speaker Recognition Evaluation (SRE).
The ongoing series of SRE yearly evaluations conducted by NIST are intended to be of interest to researchers working on the general problem of text independent speaker recognition. The task of the 2006 SRE evaluation was speaker detection, that is, to determine whether a specified speaker is speaking during a given segment of conversational telephone speech. The task was divided into 15 distinct and separate tests involving one of five training conditions and one of four test conditions. Further information about the test conditions and additional documentation is available at the NIST web site for the 2006 SRE and within the 2006 SRE Evaluation Plan.
The speech data in this release was collected by LDC as part of the Mixer project, in particular Mixer Phases 1, 2 and 3. The Mixer project supports the development of robust speaker recognition technology by providing carefully collected and audited speech from a large pool of speakers recorded simultaneously across numerous microphones and in different communicative situations and/or in multiple languages. The data is mostly English speech, but includes some speech in Arabic, Bengali, Chinese, Farsi, Hindi, Korean, Russian, Spanish, Thai and Urdu.
The telephone speech segments are multi-channel data collected simultaneously from a number of auxiliary microphones. The files are organized into four types: two-channel excerpts of approximately 10 seconds, two-channel conversations of approximately 5 minutes, summed-channel conversations also of approximately 5 minutes and a two-channel conversation with the usual telephone speech replaced by auxiliary microphone data in the putative target speaker channel. The auxiliary microphone conversations are also of approximately five minutes in length.
English language transcripts in .ctm format were produced using an automatic speech recognition (ASR) system.
2006 NIST Speaker Recognition Evaluation Test Set Part 1 is distributed on five DVD-ROM.
2011 Subscription Members will automatically receive two copies of this corpus. 2011 Standard Members may request a copy as part of their 16 free membership corpora. Non-members may license this data for US$2000.
(2) 2008 NIST Speaker Recognition Evaluation Supplemental Set was developed by LDC and National Institute of Standards and Technology (NIST) and contains additional data distributed after the main 2008 Speaker Recognition Evaluation (SRE). Specifically, the corpus consists of 770 hours of English microphone speech along with transcripts and other materials used as supplemental data in the 2008 NIST Speaker Recognition Evaluation (SRE) and in a follow-up evaluation to SRE08.
The 2008 evaluation was distinguished from prior evaluations by including not only conversational telephone speech data but also conversational speech data of comparable duration recorded over a microphone channel involving an interview scenario. The follow-up evaluation focused on speaker detection in the context of conversational interview type speech and was designed to measure the performance of SRE08 systems in previously unexposed test segment channel conditions.
LDC previously released the main 2008 NIST SRE Evaluation in three parts as 2008 NIST Speaker Recognition Evaluation Training Set Part 1 LDC2011S05, 2008 NIST Speaker Recognition Evaluation Training Set Part 2 LDC2011S07 and 2008 NIST Speaker Recognition Evaluation Test Set LDC2011S08.
The speech data in this release was collected in 2007 by LDC at its Human Subjects Data Collection Laboratories in Philadelphia and by the International Computer Science Institute (ICSI) at the University of California, Berkeley. This collection was part of the Mixer 5 project, which was designed to support the development of robust speaker recognition technology by providing carefully collected and audited speech from a large pool of speakers recorded simultaneously across numerous microphones and in different communicative situations and/or in multiple languages. Mixer participants were native English and bilingual English speakers. The microphone speech in this corpus is in English and consists of approximately 3 minute and 30 minute interview excerpts.
This supplemental data is split into four different parts which provide:
new training data distributed to 2008 SRE participants additional data distributed to participants in the 2008 SRE follow-up evaluation interviewer channel files for the 2008 SRE main test (released after the evaluations) supplemental training data (released after the evaluations) English language transcripts in .cfm format were produced using an automatic speech recognition (ASR) system and are included for some, but not all, speech data.
2008 NIST Speaker Recognition Evaluation Supplemental Set is distributed on five DVD-ROM.
2011 Subscription Members will automatically receive two copies of this corpus. 2011 Standard Members may request a copy as part of their 16 free membership corpora. Non-members may license this data for US$2000. --------------------------------------------------------------------------------
Ilya Ahtaridis Membership Coordinator -------------------------------------------------------------------- Linguistic Data Consortium Phone: 1 (215) 573-1275 University of Pennsylvania Fax: 1 (215) 573-2175 3600 Market St., Suite 810 ldc@ldc.upenn.edu Philadelphia, PA 19104 USA http://www.ldc.upenn.edu]]>http://www.luweixmu.com/home/default.asp?id=181<![CDATA[New Publications of Linguistic Data Consortium]]>adminhttp://www.luweixmu.com/home/luwig@xmu.edu.cn2011-11-17T10:03:21+08:002011-11-17T10:03:21+08:00LDC2011S09 2006 NIST Speaker Recognition Evaluation Training Set LDC2011V06 2006 NIST/USF Evaluation Resources for the VACE Program - Meeting Data Test Set Part 2 LDC2011T13 Chinese Gigaword Fifth Edition -------------------------------------------------------------------------------- Spring 2012 LDC Data Scholarship Program
Applications are now being accepted through January 15, 2012 for the Spring 2012 LDC Data Scholarship program! The LDC Data Scholarship program provides university students with access to LDC data at no-cost. This program is open to students pursuing both undergraduate and graduate studies in an accredited college or university. LDC Data Scholarships are not restricted to any particular field of study; however, students must demonstrate a well-developed research agenda and a bona fide inability to pay. The selection process is highly competitive. The application consists of two parts: (1) Data Use Proposal. Applicants must submit a proposal describing their intended use of the data. The proposal must contain the applicant's name, university, and field of study. The proposal should state which data the student plans to use and contain a description of their research project. Applicants should consult the LDC Corpus Catalog for a complete list of data distributed by LDC. Due to certain restrictions, a handful of LDC corpora are restricted to members of the Consortium. Applicants are advised to select a maximum of one to two data sets; students may apply for additional data sets during the following cycle once they have completed processing of the initial data sets and publish or present work in some juried venue. (2) Letter of Support. Applicants must submit one letter of support from their thesis adviser or department chair. The letter must confirm that the department or university lacks the funding to pay the full Non-member Fee for the data and verify the student's need for data. For further information on application materials and program rules, please visit the LDC Data Scholarship page. Students can email their applications to the LDC Data Scholarship program. Decisions will be sent by email from the same address. The deadline for the Spring 2012 program cycle is January 15, 2012.
Publications for MY2012 are still being planned; here are the working titles of data sets we intend to provide: · ARRAU 1.2 (Anaphor Resolution and Underspecification) · TORGO Dysarthic Speech · Arabic Treebank BN (broadcast news) · GALE data – all phases and tasks · Digital Archive of Southern Speech · Chinese Dependency Treebank
In addition to receiving new publications, current year members of the LDC also enjoy the benefit of licensing older data at reduced costs; current year for-profit members may use most data for commercial applications. This past year, LDC members who joined early or kept their membership current saved almost US$70,000 collectively on membership fees. Be sure to keep an eye on your mail - all previous and current LDC members will be sent an invitation to join letter and renewal invoice for MY2012. Renew early for MY2012 to save today!
New Publications (1) 2006 NIST Speaker Recognition Evaluation Training Set was developed by LDC and NIST (National Institute of Standards and Technology). It contains 595 hours of conversational telephone speech in English, Arabic, Bengali, Chinese, Hindi, Korean, Russian, Thai and Urdu and associated English transcripts used as training data in the NIST-sponsored 2006 Speaker Recognition Evaluation (SRE). The ongoing series of SRE yearly evaluations conducted by NIST are intended to be of interest to researchers working on the general problem of text independent speaker recognition. The task of the 2006 SRE evaluation was speaker detection, that is, to determine whether a specified speaker is speaking during a given segment of conversational telephone speech. The task was divided into 15 distinct and separate tests involving one of five training conditions and one of four test conditions. Further information about the test conditions and additional documentation is available at the NIST web site for the 2006 SRE and within the 2006 SRE Evaluation Plan. The speech data in this release was collected by LDC as part of the Mixer project, in particular Mixer Phases 1, 2 and 3. The Mixer project supports the development of robust speaker recognition technology by providing carefully collected and audited speech from a large pool of speakers recorded simultaneously across numerous microphones and in different communicative situations and/or in multiple languages. The data is mostly English speech, but includes some speech in the above languages The telephone speech segments are multi-channel data collected simultaneously from a number of auxiliary microphones. The files are organized into three types: two-channel excerpts of approximately 10 seconds, two-channel conversations of approximately 5 minutes and summed-channel conversations also of approximately 5 minutes. English language transcripts in .ctm format were produced using an automatic speech recognition (ASR) system. 2006 NIST Speaker Recognition Evaluation Training Set is distributed on seven DVD-ROM. 2011 Subscription Members will automatically receive two copies of this corpus. 2011 Standard Members may request a copy as part of their 16 free membership corpora. Non-members may license this data for US$2000.
(2) 2006 NIST/USF Evaluation Resources for the VACE Program - Meeting Data Test Set Part 2 was developed by researchers at the Department of Computer Science and Engineering, University of South Florida (USF), Tampa, Florida and the Multimodal Information Group at the National Institute of Standards and Technology (NIST). It contains approximately twenty hours of meeting room video data collected in 2005 and 2006 and annotated for the VACE (Video Analysis and Content Extraction) 2006 face and person tracking tasks. The VACE program was established to develop novel algorithms for automatic video content extraction, multi-modal fusion, and event understanding. During VACE Phases I and II, the program made significant progress in the automated detection and tracking of moving objects including faces, hands, people, vehicles and text in four primary video domains: broadcast news, meetings, street surveillance, and unmanned aerial vehicle motion imagery. Initial results were also obtained on automatic analysis of human activities and understanding of video sequences. Three performance evaluations were conducted under the auspices of the VACE program between 2004 and 2007. In 2006, the VACE program and the European Union's Computers in the Human Interaction Loop (CHIL) collaborated to hold the Classification of Events, Activities and Relationships (CLEAR) Evaluation. This was an international effort to evaluate systems designed to analyze people, their identities, activities, interactions and relationships in human-human interaction scenarios, as well as related scenarios. The VACE program contributed the evaluation infrastructure (e.g., data, scoring, tools) for a specific set of tasks, and the CHIL consortium, coordinated by the Karlsruhe Institute of Technology, contributed a separate set of evaluation infrastructure. To the extent possible, the VACE and CHIL programs harmonized their evaluation protocols and metrics. The meeting room data used for the 2006 test set was collected by the following sites in 2005 and 2006: Carnegie Mellon University (USA), University of Edinburgh (Scotland), IDIAP Research Institute (Switzerland), NIST (USA), Netherlands organization for Applied Scientific Research (Netherlands) and Virginia Polytechnic Institute and State University (USA). 2006 NIST/USF Evaluation Resources for the VACE Program - Meeting Data Test Set Part 2 is distributed on ten DVD-ROM. 2011 Subscription Members will automatically receive two copies of this corpus. 2011 Standard Members may request a copy as part of their 16 free membership corpora. Non-members may license this data for US$2500.
(3) Chinese Gigaword Fifth Edition was produced by LDC. It is a comprehensive archive of newswire text data that has been acquired from Chinese news sources by LDC at the University of Pennsylvania. Chinese Gigaword Fifth Edition includes all of the content of the fourth edition of Chinese Gigaword (LDC2009T27) plus new data covering the period from January 2009 through December 2010. Eight distinct sources of Chinese newswire are represented here: · Agence France Presse(afp_cmn) · Central News Agency, Taiwan(cna_cmn) · Central News Service(cns_cmn) · Guangming Daily(gmw_cmn) · People's Daily(pda_cmn) · People's Liberation Army Daily(pla_cmn) · Xinhua News Agency(xin_cmn) · Zaobao Newspaper(zbn_cmn) The seven-letter codes in the parentheses above are used for the directory names and data files for each source. Articles covering the period from January 2009 through December 2010 have been added to the Agence France Presse, Central News Agency (CNA), Central News Service, Guangming Daily, People's Liberation Army Daily and Xinhua News Agency data sets. The data from People's Daily covers the period from late June 2009 through December 2010. No new data from Zaobao has been added. Chinese Gigaword Fifth Edition is distributed on one DVD-ROM. 2011 Subscription Members will automatically receive two copies of this corpus. 2011 Standard Members may request a copy as part of their 16 free membership corpora. Non-members may license this data for US$6000. -------------------------------------------------------------------------------- Ilya Ahtaridis Membership Coordinator -------------------------------------------------------------------- Linguistic Data Consortium Phone: 1 (215) 573-1275 University of Pennsylvania Fax: 1 (215) 573-2175 3600 Market St., Suite 810 ldc@ldc.upenn.edu Philadelphia, PA 19104 USA http://www.ldc.upenn.edu]]>http://www.luweixmu.com/home/default.asp?id=180<![CDATA[New Publications of Linguistic Data Consortium]]>adminhttp://www.luweixmu.com/home/luwig@xmu.edu.cn2011-10-25T10:07:05+08:002011-10-25T10:07:05+08:00 LDC2011S08 2008 NIST Speaker Recognition Evaluation Test Set LDC2011T11 Arabic Gigaword Fifth Edition LDC2011T12 Spanish Gigaword Third Edition -------------------------------------------------------------------------------- Fall 2011 LDC Data Scholarships recipients
LDC is pleased to announce the student recipients of the Fall 2011 LDC Data Scholarship program! The LDC Data Scholarship program provides university students with access to LDC data at no-cost. Data scholarships are offered twice a year to correspond to the Fall and Spring semesters. Students are asked to complete an application which consists of a data use proposal and letter of support from their academic adviser. LDC received many strong applications from students attending universities across the globe. We've reviewed all the applications, and after careful consideration, we have selected four scholarship recipients! These students will receive no-cost copies of LDC data: Haris B C - Indian Institute of Technology Guwahati (India), Electronics & Electrical Engineering. Haris has been awarded a copy of 2005 NIST Speaker Recognition Evaluation Training Data (LDC2011S01) and 2005 NIST Speaker Recognition Evaluation Test Data (LDC2011S04) to evaluate the performance of a sparse representation speaker verification system. Friðjón Guðjohnsen - Reykjavik University (Iceland), Computer Science. Friðjón has been awarded a copy of Treebank-3 (LDC99T42) to be used in the development of tagging methods to improve the accuracy of tagging Icelandic texts. Leili Javadpour - Louisiana State University (USA), Engineering Science. Leili has been awarded a copy of BBN Pronoun Coreference and Entity Type Corpus (LDC2005T33) and Message Understanding Conference (MUC) 7 (LDC2001T02) for her work in pronominal anaphora resolution. Jad Makhlouta - American University of Beirut (Lebanon), Electrical and Computer Engineering. Jad has been awarded a copy of LDC Standard Arabic Morphological Analyzer (SAMA) Version 3.1 (LDC2010L01) for his work in Arabic text mining. Please join us in congratulating our student recipients! Look for our upcoming announcements about the submissions deadlines for the Spring 2012 LDC Data Scholarship program.
LDC data - now on Blu-ray! LDC is pleased to announce that the Blu-ray revolution has come to linguistic data! We are now offering select databases on Blu-ray Disc (BD). With BD, we'll be able to distribute some of our larger databases using a smaller number of discs. BDs also have the potential to be read more quickly than DVDs, which means faster access to data. To introduce our Blu-ray option, we would like to announce that the following databases will now be distributed on BD in addition to DVD-ROM: LDC2011T07 English Gigaword Fifth Edition, now available on 1 Blu-ray Disc LDC2006T13 Web 1T 5-gram Version 1, now available on 2 Blu-ray Discs organizations with licenses to English Gigaword Fifth Edition will be given the opportunity to 'swap' their DVDs for BDs. New licensees for Web 1T 5-gram have the option to select BD or DVD media. We expect to extend the BD option over time to other corpora in the catalog and to new releases.
LDC at NWAV 2011 NWAV’s 40th Anniversary Conference will be hosted by Georgetown University from October 27-30 and LDC will be on-hand to celebrate! Please stop by the LDC exhibition at any point during the main conference and be sure to attend LDC’s pre-conference workshop on “Demographic Coding for Sociolinguistic Corpus Archive Preparation” from 4.00 – 6.00 pm on Thursday, October 27. This workshop will be hosted by LDC Executive Director Christopher Cieri and Malcah Yaeger-Dror of the University of Arizona. It has two stated goals: 1) to catalog the need for more detailed demographic categories based on field experience that other researchers can and should exploit both for their own immediate analyses, and to facilitate sharing among research groups, and 2) to encourage the use of a core set of demographic metadata coding options. NWAV registration options can be found here. We hope to see you there! Please visit LDC’s Facebook page to follow our conference activities.
New publications (1) 2008 NIST Speaker Recognition Evaluation Test Set was developed by LDC and NIST (National Institute of Standards and Technology). It contains 942 hours of multilingual telephone speech and English interview speech along with transcripts and other materials used as test data in the 2008 NIST Speaker Recognition Evaluation (SRE). NIST SRE is part of an ongoing series of evaluations conducted by NIST. They are intended to be of interest to all researchers working on the general problem of text independent speaker recognition. The 2008 evaluation was distinguished from prior evaluations, in particular those in 2005 and 2006, by including not only conversational telephone speech data but also conversational speech data of comparable duration recorded over a microphone channel involving an interview scenario. LDC previously released the 2008 NIST SRE Training Set in two parts as LDC2011S05 and LDC2011S07. The speech data in this release was collected in 2007 by LDC at its Human Subjects Data Collection Laboratories in Philadelphia and by the International Computer Science Institute (ICSI) at the University of California, Berkeley. This collection was part of the Mixer 5 project, which was designed to support the development of robust speaker recognition technology by providing carefully collected and audited speech from a large pool of speakers recorded simultaneously across numerous microphones and in different communicative situations and/or in multiple languages. Mixer participants were native English and bilingual English speakers. The telephone speech in this corpus is predominantly English, but also includes the above languages. All interview segments are in English. Telephone speech represents approximately 368 hours of the data, whereas microphone speech represents the other 574 hours. English language transcripts in .cfm format were produced using an automatic speech recognition (ASR) system. 2008 NIST Speaker Recognition Evaluation Test Set is distributed on 9 DVD-ROM. 2011 Subscription Members will automatically receive two copies of this corpus. 2011 Standard Members may request a copy as part of their 16 free membership corpora. Non-members may license this data for US$2000.
(2) Arabic Gigaword Fifth Edition is a comprehensive archive of newswire text data that has been acquired from Arabic news sources over several years by LDC. Arabic Gigaword Fifth Edition includes all of the content of the fourth edition of Arabic Gigaword (LDC2009T30) plus new data covering the period from January 1, 2009 through December 31, 2010. Nine distinct sources of Arabic newswire are represented in this distribution: Asharq Al-Awsat (aaw_arb) Agence France Presse (afp_arb) Al-Ahram (ahr_arb) Assabah (asb_arb) Al Hayat (hyt_arb) An Nahar (nhr_arb) Al-Quds Al-Arabi (qds_arb) Ummah Press (umh_arb) Xinhua News Agency (xin_arb) The seven-character codes shown above represent both the directory names where the data files are found, and the 7-letter prefix that appears at the beginning of every file name. The 7-letter codes consist of the three-character source name IDs and the three-character language code ("arb") separated by an underscore ("_") character. The three-character language code conforms to the ISO 639-3 standard. In addition to adding new data, the following updates were made: Repeated documents in Asharq Al-Awsat data from 2008 were removed. Document formatting and docid duplication problems were corrected in Agence France Presse data. Significant duplication of content in 2007-2008 An Nahar data was detected, and the duplicated documents were removed. Arabic Gigaword Fifth Edition is distributed on 1 DVD-ROM. 2011 Subscription Members will automatically receive two copies of this corpus. 2011 Standard Members may request a copy as part of their 16 free membership corpora. Non-members may license this data for US$6000.
(3) Spanish Gigaword Third Edition was produced by LDC. It is a comprehensive archive of Spanish newswire text data that has been acquired over several years by LDC. Spanish Gigaword Third Edition includes all of the content of the second edition (LDC2009T21) and adds data collected from January 1, 2009 through December 31, 2010. The three distinct international sources of Spanish newswire in this edition, and the time spans of collection covered for each, are as follows: Agence France-Presse, Spanish (afp_spa) May 1994 - Dec 2010 Associated Press, Spanish (apw_spa) Nov 1993 - Dec 2010 Xinhua News Agency, Spanish (xin_spa) Sep 2001 - Dec 2010 The seven-letter codes in the parentheses above include the three-character source name abbreviations and the three-character language code ("spa") separated by an underscore ("_") character. The three-letter language code conforms to LDC's internal convention based on the ISO 639-3 standard. All text data are presented in SGML/XML form, using a very simple, minimal markup structure; all text consists of printable ASCII, whitespace, and printable code points in the "Latin1 Supplement" character table, as defined by both ISO-8859-1 and the Unicode Standard (ISO 10646) for the "accented" characters used in Spanish. The Supplement/accented characters are rendered using UTF-8 encoding. Spanish Gigaword Third Edition is distributed on 1 DVD-ROM. 2011 Subscription Members will automatically receive two copies of this corpus. 2011 Standard Members may request a copy as part of their 16 free membership corpora. Non-members may license this data for US$4500. -------------------------------------------------------------------------------- Ilya Ahtaridis Membership Coordinator -------------------------------------------------------------------- Linguistic Data Consortium Phone: 1 (215) 573-1275 University of Pennsylvania Fax: 1 (215) 573-2175 3600 Market St., Suite 810 ldc@ldc.upenn.edu Philadelphia, PA 19104 USA http://www.ldc.upenn.edu]]>http://www.luweixmu.com/home/default.asp?id=179<![CDATA[New Publications of Linguistic Data Consortium]]>adminhttp://www.luweixmu.com/home/luwig@xmu.edu.cn2011-10-25T10:06:25+08:002011-10-25T10:06:25+08:00 LDC2011S06- 2005 Spring NIST Rich Transcription (RT-05S) Evaluation Set - LDC2011S05- 2008 NIST Speaker Recognition Evaluation Training Set Part 1 - LDC2011T09- Arabic Treebank: Part 2 v 3.1 - -------------------------------------------------------------------------------- Fall 2011 LDC Data Scholarship Program Applications are now being accepted through September 15, 2011 for the Fall 2011 LDC Data Scholarship program! The LDC Data Scholarship program provides university students with access to LDC data at no-cost. During the previous two cycles of the program, LDC has awarded no-cost copies of LDC data valued at over US$25,000. This program is open to students pursuing both undergraduate and graduate studies in an accredited college or university. LDC Data Scholarships are not restricted to any particular field of study; however, students must demonstrate a well-developed research agenda and a bona fide inability to pay. The selection process is highly competitive. The application consists of two parts:
(1) Data Use Proposal. Applicants must submit a proposal describing their intended use of the data. The proposal must contain the applicant's name, university, and field of study. The proposal should state which data the student plans to use and contain a description of their research project. Applicants should consult the LDC Corpus Catalog for a complete list of data distributed by LDC. Due to certain restrictions, a handful of LDC corpora are restricted to members of the Consortium. Applicants are advised to select a maximum of one to two datasets; students may apply for additional datasets during the following cycle once they have completed processing of the initial datasets and publish or present work in some juried venue.
(2) Letter of Support. Applicants must submit one letter of support from their thesis adviser or department chair. The letter must confirm that the department or university lacks the funding to pay the full Non-member Fee for the data and verify the student's need for data. For further information on application materials and program rules, please visit the LDC Data Scholarship page. Students can email their applications to the LDC Data Scholarship program. Decisions will be sent by email from the same address. The deadline for the Fall 2011 program cycle is September 15, 2011. Checking in with previous LDC Data Scholarship recipients LDC introduced the Data Scholarship program during the Fall 2010 semester. Since that time, more than fifteen individual students and student research groups have been awarded no-cost copies of LDC data for their research endeavors. Here is an update on the work of a few of our student recipients: Zachary Brooks - University of Arizona (USA), PhD Candidate, Second Language Acquisition and Teaching. Zachary and his research group were awarded a copy of ECI Multilingual Text (LDC94T5) for research in eye movement tracking by native and non-natives readers. Zachary used the ECI Multilingual Text data to test how second language readers process high and low frequency words in German. The results thus far show that processing a low frequency word can make it harder to process words that come next. The group's bilingual reading processes research is ongoing and Zachary anticipates the need to utilize additional speech and text corpora for future work. Benjamin Martinez Elizalde - Monterrey Institute of Technology and Superior Studies, ITESM (Mexico), graduate student, Computer Science. Benjamin was awarded a copy of Switchboard-1 Release 2 (LDC97S62) and 2002 NIST Speaker Recognition Evaluation (SRE) (LDC2004S04) to support his research in speaker verification modeling. Benjamin's group has prepared a robust Universal Background Model (UBM) and will use the Switchboard and 2002 NIST SRE data to run enrollment and test experiments once a lower baseline is achieved. The Switchboard and SRE data will also be used to prepare the system for the 2012 NIST SRE. Xiaohui Huang - Harbin Institute of Technology (China), Shenzhen Graduate School. Xiaohui and his research group were awarded a copy of TDT5 Topics and Annotations (LDC2006T19) for his work in topic detection and tracking for large-scale web data. Xiaohui extracted 607 documents from TDT5 Multilingual Text (LDC2006T18) and designed a new clustering approach for this data set. TDT5 Topics and Annotations (LDC2006T19 ) was used to label for measuring the precision of clustering. Xiaohui next compared his clustering approach with other text clustering approaches such as k-means and agglomerative hierarchical clustering and was able to achieve good performance. Since his group's method has been validated on small test data sets, next they will look to validate the system using larger text databases and time-series databases. Muhua Zhu - Northeastern University (China), graduate student, Natural Language Processing. Muhua was awarded a copy of Chinese Treebank (CTB) 7.0 (LDC2010T07) to support the development of a high-accuracy Chinese parser. Currently, Muhua is writing a survey paper on Chinese syntactic parsing which studies the performance of different parsing models on the versions of LDC's CTBs. Muhua had expected that parsing accuracy would increase with the additional data from CTB7.0, but accuracy decreased in some instances perhaps because of the inclusion of web text in CTB 7.0. Muhua next plans to use re-ranking methods for syntactic parsing and to extract a Combinatory Categorial Grammar bank (CCG bank) from CTB7.0. We'd like to thanks these students for providing an update on their research. Stay tuned for further reports from other data scholarship recipients. Weizmann Institute students are introduced to LDC data
LDC data was featured in an introductory speech recognition course at the Weizmann Institute of Science in Rehovot, Israel. Visiting professor, Karen Livescu, of Toyota Technological Institute at Chicago and University of Chicago, Department of Computer Science used several LDC corpora, including CSR-I (WSJ0) Complete (LDC93S6A), Switchboard-1 Release 2 (LDC97S62), TIDIGITS (LDC93S10), and TIMIT Acoustic-Phonetic Continuous Speech Corpus (LDC93S1) for homework and term projects, with a few examples shown during in-class demonstration. The students enrolled in the course were computer science and mathematics graduate students and all were new to automatic speech recognition (ASR). They had backgrounds in probability, but no significant experience with the probabilistic models used in ASR, such as hidden Markov models and Gaussian mixtures. Livescu provided baseline recognizers that the students could modify, so that even beginning students could focus on specific components, while using real data with results in the literature to compare against. Since the students were provided with real data that the research community actively uses, students were motivated by the potential for 'real' results if their projects went as planned. As Livescu noted, 'while starting out in ASR from scratch is very difficult, the availability of toolkits and LDC data makes it possible for students in an introductory class to do productive research quite quickly'. Many thanks to Karen Livescu for sharing an example of how LDC data can be used for teaching purposes.
LDC Exhibiting at Interspeech 2011, Florence Italy LDC is returning to Europe to participate in Interspeech 2011. The conference will be held from August 28-31 at the Firenze Fiera, conveniently located near the Stazione di Santa Maria Novella. Please stop by LDC’s exhibition booth to say hello and learn more about current happenings at the Consortium. Interspeech 2011’s theme is ‘Speech Science and Technology for Real Life’. You may learn more about the conference here. The main conference will feature keynotes on the following topics: Speaking More Like You: Entrainment in Conversational Speech, Prof. Julia Hirschberg Neural Representations of Word Meanings, Prof. Tom Mitchell Honest Signals, Prof. Sandy Pentland Conference organizers have also scheduled a roundtable discussion for August 31st on ‘Future and Applications of Speech and Language Technologies for the Good Health of Society’ which will be led by Profs. Gabriele Miceli, Björn Granström and Hiroshi Ishiguro. You are encouraged to keep track of LDC’s Interspeech preparations on our Facebook page. We hope to see you there!
New Publications (1) 2005 Spring NIST Rich Transcription (RT-05S) Conference Meeting Evaluation Set was developed by LDC and the National Institute of Standards and Technology (NIST). It contains approximately 78 hours of English meeting speech, reference transcripts and other material used in the RT Spring 2005 evaluation. Rich Transcription (RT) is broadly defined as a fusion of speech-to-text (STT) technology and metadata extraction technologies providing the bases for the generation of more usable transcriptions of human-human speech in meetings. RT-05S included the following tasks in the meeting domain: Speech-To-Text (STT) - convert spoken words into streams of text Speaker Diarization (SPKR) - find the segments of time within a meeting in which each meeting participant is talking Speech Activity Detection (SAD) - detect when someone in a meeting space is talking Further information about the evaluation is available on the RT-05 Spring Evaluation Website. The data in this release consists of portions of meeting speech collected between 2001 and 2005 by the IDIAP Research Institute's Augmented Multi-Party Interaction project (AMI), Martigny, Switzerland; International Computer Science Institute (ICSI) at University of California, Berkeley; Interactive Systems Laboratories (ISL) at Carnegie Mellon University (CMU), Pittsburgh, PA; NIST; and Virginia Polytechnic Institute and State University (VT), Blacksburg, VA. Each meeting excerpt contains a head-mic recording for each subject and one or more distant microphone recordings. Reference transcripts for the evaluation excerpts were prepared by LDC according to its Meeting Recording Careful Transcription Guidelines. Those specifications are designed to provide an accurate, verbatim (word-for-word) transcription, time-aligned with the audio file and including the identification of additional audio and speech signals with special mark-up. 2005 Spring NIST Rich Transcription (RT-05S) Conference Meeting Evaluation Set is distributed on 3 DVD-ROM. 2011 Subscription Members will automatically receive two copies of this corpus. 2011 Standard Members may request a copy as part of their 16 free membership corpora. Non-members may license this data for $2250.
(2) 2008 NIST Speaker Recognition Evaluation Training Set Part 1 was developed by LDC and the National Institute of Standards and Technology (NIST). It contains 640 hours of multilingual telephone speech and English interview speech along with transcripts and other materials used as training data in the 2008 NIST Speaker Recognition Evaluation (SRE). SRE is part of an ongoing series of evaluations conducted by NIST. These evaluations are an important contribution to the direction of research efforts and the calibration of technical capabilities. They are intended to be of interest to all researchers working on the general problem of text independent speaker recognition. The 2008 evaluation was distinguished from prior evaluations, in particular those in 2005 and 2006, by including not only conversational telephone speech data but also conversational speech data of comparable duration recorded over a microphone channel involving an interview scenario. The speech data in this release was collected in 2007 by LDC at its Human Subjects Data Collection Laboratories in Philadelphia and by the International Computer Science Institute (ICSI) at the University of California, Berkley. This collection was part of the Mixer 5 project, which was designed to support the development of robust speaker recognition technology by providing carefully collected and audited speech from a large pool of speakers recorded simultaneously across numerous microphones and in different communicative situations and/or in multiple languages. Mixer participants were native English and bilingual English speakers. The telephone speech in this corpus is predominately English; all interview segments are in English. Telephone speech represents approximately 565 hours of the data, where as microphone speech represents the other 75 hours. The telephone speech segments include excerpts in the range of 8-12 seconds and 5 minutes from longer original conversations. The interview material includes short conversation interview segments of approximately 3 minutes from a longer interview session. English language transcripts in .cfm format were produced using an automatic speech recognition (ASR) system. 2008 NIST Speaker Recognition Evaluation Training Set Part 1 is distributed on 9 DVD-ROM. 2011 Subscription Members will automatically receive two copies of this corpus. 2011 Standard Members may request a copy as part of their 16 free membership corpora. Non-members may license this data for $2000.
(3) Arabic Treebank: Part 2 (ATB2) v 3.1 was developed at LDC. It consists of 501 newswire stories from Ummah Press with part-of-speech (POS), morphology, gloss and syntactic treebank annotation in accordance with the Penn Arabic Treebank (PATB) Guidelines developed in 2008 and 2009. This release represents a significant revision of LDC's previous ATB2 publication: Arabic Treebank: Part 2 v 2.0 LDC2004T02. The ongoing PATB project supports research in Arabic-language natural language processing and human language technology development. The methodology and work leading to the release of this publication are described in detail in the documentation accompanying this corpus and in two research papers: Enhancing the Arabic Treebank: A Collaborative Effort toward New Annotation Guidelines and Consistent and Flexible Integration of Morphological Annotation in the Arabic Treebank. ATB2 v 3.1 contains a total of 144,199 source tokens before clitics are split, and 169,319 tree tokens after clitics are separated for the treebank annotation. Source texts were selected from Ummah Press news archives covering the period from July 2001 through September 2002. Arabic Treebank: Part 2 v 3.1 is distributed via web download. 2011 Subscription Members will automatically receive two copies of this corpus on disc. 2011 Standard Members may request a copy as part of their 16 free membership corpora. Non-members may license this data for $4500. -------------------------------------------------------------------------------- Ilya Ahtaridis Membership Coordinator -------------------------------------------------------------------- Linguistic Data Consortium Phone: 1 (215) 573-1275 University of Pennsylvania Fax: 1 (215) 573-2175 3600 Market St., Suite 810 ldc@ldc.upenn.edu Philadelphia, PA 19104 USA http://www.ldc.upenn.edu]]>http://www.luweixmu.com/home/default.asp?id=178<![CDATA[New Publications of Linguistic Data Consortium]]>adminhttp://www.luweixmu.com/home/luwig@xmu.edu.cn2011-07-18T21:33:29+08:002011-07-18T21:33:29+08:00- LDC Sponsors a Student Group at 2011 International Linguistics Olympiad - - LDC Receives META Prize from META-NET -
New publications: - 2005 NIST Speaker Recognition Evaluation Test Data - - 2006 NIST Spoken Term Detection Evaluation Set - - NIST/USF Evaluation Resources for the VACE Program - Meeting Data Test Set Part 2 - -------------------------------------------------------------------------------- LDC Sponsors a Student Group at 2011 International Linguistics Olympiad LDC is happy to support the 2011 International Linguistics Olympiad by sponsoring a student team. The IOL is one of the twelve International Science Olympiads and is an annual event that brings together students from around the world to compete in linguistically–based challenges. This year’s competition takes place from July 24-30 at Carnegie Mellon University, Pittsburgh, PA USA. Students do not need to have a background in linguistics in order to participate since they typically use analysis and deductive reasoning to solve the competition problems. Please visit the 2011 IOL website for additional details. We wish good luck to all of the participants!
LDC Receives META Prize from META-NET LDC was awarded a ‘2nd META Prize’ from META-NET ‘for outstanding long term commitment to the preparation and distribution of language resources and technologies.’ The META Prize is awarded by META-NET to those who provide outstanding products or services that support the European Multilingual Information Society. META-NET is a Network of Excellence dedicated to fostering the technological foundations of a multilingual European information society. Several organizations were honored at this year’s META Forum in Budapest; LDC and ELRA were both honored for supporting and developing language resources.
New Publications
(1) 2005 NIST Speaker Recognition Evaluation Test Data was developed at LDC and NIST (National Institute of Standards and Technology). It consists of 525 hours of conversational telephone speech in English, Arabic, Mandarin Chinese, Russian and Spanish and associated English transcripts used as test data in the NIST-sponsored 2005 Speaker Recognition Evaluation (SRE). The ongoing series of SRE yearly evaluations conducted by NIST are intended to be of interest to researchers working on the general problem of text independent speaker recognition. To that end the evaluations are designed to be simple, to focus on core technology issues, to be fully supported and accessible. The task of the 2005 SRE evaluation was speaker detection, that is, to determine whether a specified speaker is speaking during a given segment of conversational speech. The task was divided into 20 distinct and separate tests involving one of five training conditions and one of four test conditions. Further information about the task conditions is contained in the The NIST Year 2005 Speaker Recognition Evaluation Plan. The speech data consists of conversational telephone speech with "multi-channel" data collected by LDC simultaneously from a number of auxiliary microphones. The files are organized into two segments: 10 second two-channel excerpts (continuous segments from single conversations that are estimated to contain approximately 10 seconds of actual speech in the channel of interest) and 5 minute two-channel conversations. The data are stored as 8-bit u-law speech signals in NIST SPHERE format. In addition to the standard header fields, the SPHERE header for each file contains some auxiliary information that includes the language of the conversation and whether the data was recorded over a telephone line. English language word transcripts in .cmt format were produced using an automatic speech recognition system (ASR) with error rates in the range of 15-30%. 2005 NIST Speaker Recognition Evaluation Test Data is distributed on 7 DVD-ROM. 2011 Subscription Members will automatically receive two copies of this corpus. 2011 Standard Members may request a copy as part of their 16 free membership corpora. Non-members may license this data for US$2000.
(2) 2006 NIST Spoken Term Detection Evaluation Set was compiled by researchers at NIST (National Institute of Standards and Technology) and contains approximately eighteen hours of Arabic, Chinese and English broadcast news, English conversational telephone speech and English meeting room speech used in NIST's 2006 Spoken Term Detection (STD) evaluation. The STD initiative is designed to facilitate research and development of technology for retrieving information from archives of speech data with the goals of exploring promising new ideas in spoken term detection, developing advanced technology incorporating these ideas, measuring the performance of this technology and establishing a community for the exchange of research results and technical insights. The 2006 STD task was to find all of the occurrences of a specified "term" (a sequence of one or more words) in a given corpus of speech data. The evaluation was intended to develop technology for rapidly searching very large quantities of audio data. Although the evaluation used modest amounts of data, it was structured to simulate the very large data situation and to make it possible to extrapolate the speed measurements to much larger data sets. Therefore, systems were implemented in two phases: indexing and searching. In the indexing phase, the system processes the speech data without knowledge of the terms. In the searching phase, the system uses the terms, the index, and optionally the audio to detect term occurrences. The evaluation corpus consists of three data genres: broadcast news (BNews), conversational telephone speech (CTS) and conference room meetings (CONFMTG). The broadcast news material was collected in 2003 and 2004 by LDC's broadcast collection system from the following sources: ABC (English), Aljazeera (Arabic), China Central TV (Chinese), CNN (English), CNBC (English), Dubaie TV (Arabic), New Tang Dynasty TV (Chinese), Public Radio International (English) and Radio Free Asia(Chinese). The CTS data was taken from the Switchboard data sets (e.g., Switchboard-2 Phase 1 LDC98S75, Switchboard-2 Phase 2 LDC99S79) and the Fisher corpora (e.g., Fisher English Training Speech Part 1 LDC2004S13), also collected by LDC. The conference room meeting material consists of goal-oriented, small group round table meetings and was collected in 2004 and 2005 by NIST, the International Computer Science Institute (Berkeley, California), Carnegie Mellon University (Pittsburgh, PA), TNO (The Netherlands) and Virginia Polytechnic Institute and State University (Blacksburg, VA) as part of the AMI corpus project. This evaluation corpus includes scoring software. It uses the inputs described in the STD Evaluation plan to complete the evaluation of a system. Each BNews recording is a 1-channel, pcm-encoded, 16Khz, SPHERE formatted file. CTS recordings are 2-channel, u-law encoded, 8 Khz, SPHERE formatted files. The CONFMTG files contain a single recorded channel. 2006 NIST Spoken Term Detection Evaluation Set is distributed on 1 DVD-ROM. 2011 Subscription Members will automatically receive two copies of this corpus. 2011 Standard Members may request a copy as part of their 16 free membership corpora. Non-members may license this data for US$800.
(3) NIST/USF Evaluation Resources for the VACE Program - Meeting Data Test Set Part 2 was developed by researchers at the Department of Computer Science and Engineering, University of South Florida (USF), Tampa, Florida and the Multimodal Information Group at the National Institute of Standards and Technology (NIST). It contains approximately thirteen hours of meeting room video data collected in 2001 and 2002 at NIST's Meeting Data Collection Laboratory and used in the VACE (Video Analysis and Content Extraction) 2005 evaluation. The VACE program was established to develop novel algorithms for automatic video content extraction, multi-modal fusion, and event understanding. During VACE Phases I and II, the program made significant progress in the automated detection and tracking of moving objects including faces, hands, people, vehicles and text in four primary video domains: broadcast news, meetings, street surveillance, and unmanned aerial vehicle motion imagery. Initial results were also obtained on automatic analysis of human activities and understanding of video sequences. Three performance evaluations were conducted under the auspices of the VACE program between 2004 and 2007. The 2005 evaluation was administered by USF in collaboration with NIST and guided by an advisory forum including the evaluation participants. LDC has previously released NIST/USF Evaluation Resources for the VACE Program -- Meeting Data Training Set Part 1 LDC2011V01, NIST/USF Evaluation Resources for the VACE Program -- Meeting Data Training Set Part 2 LDC2011V02 and NIST/USF Evaluation Resources for the VACE Program -- Meeting Data Test Set Part 1 LDC2011V03. NIST's Meeting Data Collection Laboratory is designed to collect corpora to support research, development and evaluation in meeting recognition technologies. It is equipped to look and sound like a conventional meeting space. The data collection facility includes five Sony EV1-D30 video cameras, four of which have stationary views of a center conference table (one view from each surrounding wall) with a fixed focus and viewing angle, and an additional "floating" camera which is used to focus on particular participants, whiteboard or conference table depending on the meeting forum. The data is captured in a NIST-internal file format. The video data was extracted from the NIST format and encoded using the MPEG-2 standard in NTSC format. Further information concerning the video data parameters can found in the documentation included with this corpus. NIST/USF Evaluation Resources for the VACE Program - Meeting Data Test Set Part 2 is distributed on 8 DVD-ROM. 2011 Subscription Members will automatically receive two copies of this corpus. 2011 Standard Members may request a copy as part of their 16 free membership corpora. Non-members may license this data for US$2500. -------------------------------------------------------------------------------- Ilya Ahtaridis Membership Coordinator -------------------------------------------------------------------- Linguistic Data Consortium Phone: 1 (215) 573-1275 University of Pennsylvania Fax: 1 (215) 573-2175 3600 Market St., Suite 810 ldc@ldc.upenn.edu Philadelphia, PA 19104 USA http://www.ldc.upenn.edu]]>http://www.luweixmu.com/home/default.asp?id=177<![CDATA[New Publications of Linguistic Data Consortium]]>adminhttp://www.luweixmu.com/home/luwig@xmu.edu.cn2011-06-20T11:59:05+08:002011-06-20T11:59:05+08:00 - LDC is now on your favorite Social Networks - New Publications: LDC2011S02 - 2006 NIST Spoken Term Detection Development Set - LDC2011T08 - Datasets for Generic Relation Extraction (reACE) - LDC2011T07 - English Gigaword Fifth Edition -------------------------------------------------------------------------------- LDC at ACL: June 20-22 2011 ACL has returned to North America and LDC is taking this opportunity to interact with top HLT researchers in beautiful Portland, or. LDC’s exhibition table will feature information on new developments at the consortium and will also be the go-to point for exciting new, green giveaways. LDC’s Seth Kulick will be presenting research on ‘Using Derivation Trees for Treebank Error Detection’ (S-66) during Monday’s evening poster session (20 June, 6.00 – 8.30 pm). The abstract for this paper, coauthored by LDCers Ann Bies and Justin Mott, is as follows: This work introduces a new approach to checking treebank consistency. Derivation trees based on a variant of Tree Adjoining Grammar are used to compare the annotation of word sequences based on their structural similarity. This overcomes the problems of earlier approaches based on using strings of words rather than tree structure to identify the appropriate contexts for comparison. We report on the result of applying this approach to the Penn Arabic Treebank and how this approach leads to high precision of error detection. We hope to see you there. LDC is now on your favorite Social Networks (Facebook, LinkedIn and RSS, oh my!) Over the past few months, LDC has responded to requests from the community to increase our online presence. We are happy to announce that LDC now has its very own Facebook page, LinkedIn profile (independent of the University of Pennsylvania) and Blog, which provides an RSS feed for LDC newsletters. Please visit LDC on our various profiles and let us know what you think!
New Publications (1) 2006 NIST Spoken Term Detection Development Set was compiled by researchers at NIST (National Institute of Standards and Technology) and contains eighteen hours of Arabic, Chinese and English broadcast news, English conversational telephone speech and English meeting room speech used in NIST's 2006 Spoken Term Detection (STD) evaluation. The STD initiative is designed to facilitate research and development of technology for retrieving information from archives of speech data with the goals of exploring promising new ideas in spoken term detection, developing advanced technology incorporating these ideas, measuring the performance of this technology and establishing a community for the exchange of research results and technical insights. The 2006 STD task was to find all of the occurrences of a specified term (a sequence of one or more words) in a given corpus of speech data. The evaluation was intended to develop technology for rapidly searching very large quantities of audio data. Although the evaluation used modest amounts of data, it was structured to simulate the very large data situation and to make it possible to extrapolate the speed measurements to much larger data sets. Therefore, systems were implemented in two phases: indexing and searching. In the indexing phase, the system processes the speech data without knowledge of the terms. In the searching phase, the system uses the terms, the index, and optionally the audio to detect term occurrences. The development corpus consists of three data genres: broadcast news (BN), conversational telephone speech (CTS) and conference room meetings (CONFMTG). The broadcast news material was collected in 2001 by LDC's broadcast collection system from the following sources: ABC (English), China Broadcasting System (Chinese), China Central TV (Chinese), China National Radio (Chinese), China Television System (Chinese), CNN (English), MSNBC/NBC (English), Nile TV (Arabic), Public Radio International (English) and Voice of America (Arabic, Chinese, English). The CTS data was taken from the Switchboard data sets (e.g., Switchboard-2 Phase 1 LDC98S75, Switchboard-2 Phase 2 LDC99S79) and the Fisher corpora (e.g., Fisher English Training Sppech Part 1 LDC2004S13), also collected by LDC. The conference room meeting material consists of goal-oriented, small group round table meetings and was collected in 2001, 2004 and 2005 by NIST, the International Computer Science Institute (Berkeley, California), Carnegie Mellon University (Pittsburgh, PA) and Virginia Polytechnic Institute and State University (Blacksburg, VA) as part of the AMI corpus project. Each BNews recording is a 1-channel, pcm-encoded, 16Khz, SPHERE formatted file. CTS recordings are 2-channel, u-law encoded, 8 Khz, SPHERE formatted files. TheCONFMTG files contain a single recorded channel. 2006 NIST Spoken Term Detection Development Set is distributed on 1 DVD-ROM. 2011 Subscription Members will automatically receive two copies of this corpus. 2011 Standard Members may request a copy as part of their 16 free membership corpora. Non-members may license this data for US$800. (2) Datasets for Generic Relation Extraction (reACE) was developed at The University of Edinburgh, Edinburgh, Scotland. It consists of English broadcast news and newswire data originally annotated for the ACE (Automatic Content Extraction) program to which the Edinburgh Regularized ACE (reACE) mark-up has been applied. The Edinburgh relation extraction (RE) task aims to identify useful information in text (e.g., PersonW works for organisationX, GeneY encodes ProteinZ) and to recode it in a format such as a relational database or RDF triple store (a database for the storage and retrieval of Resource Description Framework (RDF) metadata) that can be more effectively used for querying and automated reasoning. A number of resources have been developed for training and evaluation of automatic systems for RE in different domains. However, comparative evaluation is impeded by the fact that these corpora use different markup formats and different notions of what constitutes a relation. reACE solves this problem by converting data to a common document type using token standoff and including detailed linguistic markup while maintaining all information in the original annotation. The subsequent re-annotation process normalizes the two data sets so that they comply with a notion of relation that is intuitive, simple and informed by the semantic web. The data in this corpus consists of newswire and broadcast news material from ACE 2004 Multilingual Training Corpus LDC 2005T09 and ACE 2005 Multilingual Training Corpus LDC2006T06 . This material has been standardized for evaluation of multi-type RE across domains. Annotation includes (1) a refactored version of the original data to a common XML document type; (2) linguistic information from LT-TTT (a system for tokenizing text and adding markup) and MINIPAR (an English parser); and (3) a normalized version of the original RE markup that complies with a shared notion of what constitutes a relation across domains. The data sources represented in the corpus were collected by LDC in 2000 and 2003 and consist of the following: ABC, Agence France Presse, Associated Press, Cable News Network, MSNBC/NBC, New York Times, Public Radio International, Voice of America and Xinhua News Agency. Datasets for Generic Relation Extraction (reACE) is distributed via web download. 2011 Subscription Members will automatically receive two copies of this corpus on disc. 2011 Standard Members may request a copy as part of their 16 free membership corpora. Non-members may license this data for US$800. (3) English Gigaword Fifth Edition is a comprehensive archive of newswire text data that has been acquired over several years by the LDC at the University of Pennsylvania. The fifth edition includes all of the contents in English Gigaword Fourth Edition (LDC2009T13) plus new data covering the 24-month period of January 2009 through December 2010. The seven distinct international sources of English newswire included in this edition are the following: Agence France-Presse, English Service (afp_eng) Associated Press Worldstream, English Service (apw_eng) Central News Agency of Taiwan, English Service (cna_eng) Los Angeles Times/Washington Post Newswire Service (ltw_eng) Washington Post/Bloomberg Newswire Service (wpb_eng) New York Times Newswire Service (nyt_eng) Xinhua News Agency, English Service (xin_eng) The seven letter codes in the parentheses above include the three-character source name abbreviations and the three-character language code ("eng") separated by an underscore ("_") character. The three-letter language code conforms to LDC's internal convention based on the ISO 639-3 standard.
Data The following table sets forth the overall totals for each source. Note that "Total-MB" refers to the quantity of date when unzipped (approximately 26 gigabytes), "Gzip-MB" refers to compressed file sizes as stored on the DVD-ROMs and "K-wrds" refers to the number of whitespace-separated tokens (of all types) after all SGML tags are eliminated: Source #Files Gzip-MB Totl-MB K-wrds #DOCs afp_eng 146 1732 4937 738322 2479624 apw_eng 193 2700 7889 1186955 3107777 cna_eng 144 86 261 38491 145317 ltw_eng 127 651 1694 268088 411032 nyt_eng 197 3280 8938 1422670 1962178 wpb_eng 12 42 111 17462 26143 xin_eng 191 834 2518 360714 1744025 TOTAL 1010 9325 26348 4032686 9876086
English Gigaword Fifth Edition is distributed on 3 DVD-ROM. 2011 Subscription Members will automatically receive two copies of this corpus. 2011 Standard Members may request a copy as part of their 16 free membership corpora. Non-members may license this data for US$6000. -------------------------------------------------------------------------------- Ilya Ahtaridis Membership Coordinator -------------------------------------------------------------------- Linguistic Data Consortium Phone: 1 (215) 573-1275 University of Pennsylvania Fax: 1 (215) 573-2175 3600 Market St., Suite 810 ldc@ldc.upenn.edu Philadelphia, PA 19104 USA http://www.ldc.upenn.edu]]>http://www.luweixmu.com/home/default.asp?id=176<![CDATA[New Publications of Linguistic Data Consortium]]>adminhttp://www.luweixmu.com/home/luwig@xmu.edu.cn2011-05-25T08:33:00+08:002011-05-25T08:33:00+08:00LDC2011S01 - NIST 2005 Speaker Recognition Evaluation Training Data - LDC2011V03 - NIST/USF Evaluation Resources for the VACE Program - Meeting Data Test Set Part 3 - -------------------------------------------------------------------------------- Early Renewing Members Save Again! Once again, LDC's early renewal discount program has resulted in significant savings for our members! For Membership Year (MY) 2011, about 120 organizations that renewed membership or joined early received a discount on their membership fees. Taken together, these members saved almost US$70,000! MY 2010 members are reminded that they are still eligible for a 5% discount when renewing. This discount will apply throughout 2011, regardless of time of renewal. By joining for MY 2011, any organization can take advantage of membership benefits including free membership year data as well as discounts on older LDC corpora. Please visit our Members FAQ for further information. LDC to Close for Memorial Day - May 30, 2011 LDC would like to inform our customers that we will be closed on Monday, May 30, 2011 in observance of the US Memorial Day holiday. Our offices will reopen on Tuesday, May 31, 2011.
New Publications
(1) 2005 NIST Speaker Recognition Evaluation Training Data was developed at LDC and NIST (National Insitute of Standards and Technology). It consists of 392 hours of conversational telephone speech in English, Arabic, Mandarin Chinese, Russian and Spanish and associated English transcripts used as training data in the NIST-sponsored 2005 Speaker Recognition Evaluation (SRE). The ongoing series of SRE yearly evaluations conducted by NIST are intended to be of interest to researchers working on the general problem of text independent speaker recognition. To that end the evaluations are designed to be simple, to focus on core technology issues, to be fully supported and to be accessible to those wishing to participate. The task of the 2005 SRE evaluation was speaker detection, that is, to determine whether a specified speaker is speaking during a given segment of conversational speech. The task was divided into 20 distinct and separate tests involving one of five training conditions and one of four test conditions. The speech data consists of conversational telephone speech with "multi-channel" data collected simultaneously from a number of auxiliary microphones. The files are organized into two segments: 10 second two-channel excerpts (continuous segments from single conversations that are estimated to contain approximately 10 seconds of actual speech in the channel of interest) and 5 minute two-channel conversations. The speech files are stored as 8-bit u-law speech signals in separate SPHERE files. In addition to the standard header fields, the SPHERE header for each file contains some auxiliary information that includes the language of the conversation and whether the data was recorded over a telephone line. English language word transcripts in .cmt format were produced using an automatic speech recognition system (ASR) and contain error rates in the range of 15-30%. 2005 NIST Speaker Recognition Evaluation Training Data is distributed on 6 DVD-ROM. 2011 Subscription Members will automatically receive two copies of this corpus. 2011 Standard Members may request a copy as part of their 16 free membership corpora. Non-members may license this data for US$2000.
(2) NIST/USF Evaluation Resources for the VACE Program - Meeting Data Test Set Part 3, Linguistic Data Consortium (LDC) catalog number LDC2011V03 and isbn 1-58563-579-0, was developed by researchers at the Department of Computer Science and Engineering, University of South Florida (USF), Tampa, Florida and the Multimodal Information Group at the National Institute of Standards and Technology (NIST). It contains approximately eleven hours of meeting room video data collected in 2001 and 2002 at NIST's Meeting Data Collection Laboratory and annotated for the VACE (Video Analysis and Content Extraction) 2005 face, person and hand detection and tracking tasks. The VACE program was established to develop novel algorithms for automatic video content extraction, multi-modal fusion, and event understanding. During VACE Phases I and II, the program made significant progress in the automated detection and tracking of moving objects including faces, hands, people, vehicles and text in four primary video domains: broadcast news, meetings, street surveillance, and unmanned aerial vehicle motion imagery. Initial results were also obtained on automatic analysis of human activities and understanding of video sequences. Three performance evaluations were conducted under the auspices of the VACE program between 2004 and 2007. The 2005 evaluation was administered by USF in collaboration with NIST and guided by an advisory forum including the evaluation participants. A summary of results of the evaluation can be found in the 2005 VACE results and analysis paper included in this release. NIST's Meeting Data Collection Laboratory is designed to collect corpora to support research, development and evaluation in meeting recognition technologies. It is equipped to look and sound like a conventional meeting space. The data collection facility includes five Sony EV1-D30 video cameras, four of which have stationary views of a center conference table (one view from each surrounding wall) with a fixed focus and viewing angle, and an additional "floating" camera which is used to focus on particular participants, whiteboard or conference table depending on the meeting forum. The data is captured in a NIST-internal file format. The video data was extracted from the NIST format and encoded using the MPEG-2 standard in NTSC format. Further information concerning the video data parameters can found in the documentation included with this corpus. NIST/USF Evaluation Resources for the VACE Program - Meeting Data Test Set Part 3 is distributed on 7 DVD-ROM. 2011 Subscription Members will automatically receive two copies of this corpus. 2011 Standard Members may request a copy as part of their 16 free membership corpora. Non-members may license this data for US$2500. -------------------------------------------------------------------------------- Ilya Ahtaridis Membership Coordinator -------------------------------------------------------------------- Linguistic Data Consortium Phone: 1 (215) 573-1275 University of Pennsylvania Fax: 1 (215) 573-2175 3600 Market St., Suite 810 ldc@ldc.upenn.edu Philadelphia, PA 19104 USA http://www.ldc.upenn.edu]]>http://www.luweixmu.com/home/default.asp?id=175