Christoph: /* External links */

2007-06-18T03:53:35Z

‎External links

Christoph at 03:32, 18 June 2007

2007-06-18T03:32:14Z

New page

This article will describe my work in developing a '''paraphrase algorithm''' using Project Gutenberg as my corpora. It is a form of natural language processing (NLP) in computational linguistics.

The idea is to first extract comparable sub-corpora from my main corpus and use this as my training set. To start with, I will first build a basic sub-subset of parallel corpus and use this for sentence clustering. I am trying to collect data for inferring templates from sentences that appear to be similar on a word-by-word level.

==Examples==
===Sem Experiment===
See: [http://www.cs.cornell.edu/Info/Projects/NLP/statpar/instr1.html Statistical Paraphrasing Project] from the [http://www.cs.cornell.edu/Info/Projects/NLP/ Cornell Natural Language Processing Group]

===Swords to ploughshares===
Let <code>A1 = Isaiah 2:4</code>, <code>B1 = Micah 4:3</code>, and <code>C1 = Joel 3:10</code>. With,
;A1: And he shall judge among the nations, and shall rebuke many people: and they shall beat their swords into plowshares, and their spears into pruninghooks: nation shall not lift up sword against nation, neither shall they learn war any more.
;B1: And he shall judge among many people, and rebuke strong nations afar off; and they shall beat their swords into plowshares, and their spears into pruninghooks: nation shall not lift up a sword against nation, neither shall they learn war any more.
;C1: Beat your plowshares into swords, and your pruninghooks into spears: let the weak say, I am strong.

*sentence clustering:
<pre>
A1a: And he shall judge among the nations,
B1a: And he shall judge among many people,

A1b: and shall rebuke many people:
B1b: and rebuke strong nations afar off;

A1c: and they shall beat their swords into plowshares,
B1c: and they shall beat their swords into plowshares,

A1d: and their spears into pruninghooks:
B1d: and their spears into pruninghooks:

A1e: nation shall not lift up sword against nation,
B1e: nation shall not lift up a sword against nation,

A1f: neither shall they learn war any more.
B1f: neither shall they learn war any more.
</pre>

*inducing patterns (arguments in square brackets):
{A1c,A1d,A1f} = {B1c,B1d,B1f}
A1a: And he shall judge among [the nations],
B1a: And he shall judge among [many people],
A1b: and [shall rebuke] [many people]:
B1b: and [rebuke] [strong nations] [afar off];
A1e: nation shall not lift up [sword] against nation,
B1e: nation shall not lift up [a sword] against nation,

==References==
*Barzilay R, Lee L (2003). "[http://www.cs.cornell.edu/Info/Projects/NLP/statpar.html Learning to Paraphrase: An Unsupervised Approach Using Multiple-Sequence Alignment]". ''Proceedings of HLT-NAACL, pp 16-23''.

==See also==
*[[wikipedia:Natural language processing]]
*[[wikipedia:Computational linguistics]]
*[[wikipedia:Corpus linguistics]]
*[[wikipedia:Part-of-speech tagging]] (POS tagging or POST also called grammatical tagging)
*[http://www.mitpressjournals.org/loi/coli?cookieSet=1 Computational Linguistics (journal)]

==External links==
*[http://www.gelbukh.com/clbook/ COMPUTATIONAL LINGUISTICS: Models, Resources, Applications] — free online book
*[http://www.cs.cornell.edu/home/llee/papers/statpar-informal.draft.html An informal explanation of "Learning to Paraphrase: An Unsupervised Approach Using Multiple-Sequence Alignment"]
*[http://ejohn.org/projects/javascript-diff-algorithm/ Javascript Diff Algorithm]

[[Category:Linguistics]]

@@ Line 57: / Line 57: @@
 *[http://www.cs.cornell.edu/home/llee/papers/statpar-informal.draft.html An informal explanation of "Learning to Paraphrase: An Unsupervised Approach Using Multiple-Sequence Alignment"]
 *[http://ejohn.org/projects/javascript-diff-algorithm/ Javascript Diff Algorithm]
 [[Category:Linguistics]]

Paraphrase algorithm - Revision history

Christoph: /* External links */

Christoph at 03:32, 18 June 2007