NAME

dispersion_and_fmod.pl - computes the dispersion, frequency, and modified frequency of entries.

SYNOPSIS

dispersion_and_fmod.pl [-h|--help --man -c|--columns] DATA

Input

DATA is a tab-delimited file with any number of fields for the entry (e.g. lemma), one field (second last) for the class (e.g. genre), and one field (last) for the frequency of that class. Could be redirected from STDIN.

Optional options

-h|--help: prints a help message to STDOUT.
--man: prints full documentation in man page format.
-c|--columns: indicates that the input file has one line of column headers (which should be skipped).

Output

A tab-delimited file with modified frequency, frequency, dispersion, and original entry fields.

DESCRIPTION

This Perl script computes the Dispersion (D), Frequency (F), and Modified Frequency (FMod) for each entry. D and FMod were introduced by Alphonse Juilland and E. Chang-Rodriguez: Frequency Dictionary of Spanish Words, 1964, Mouton & Co., The Hague, The Netherlands.

The measures are computed as follows:

FMod: FMod = F * D
F: The raw frequency.
D: D = 1 - s / ( m * ( sqrt( n - 1 ) ) ); where; s = standard deviance; m = mean (occurrence); n = number of classes
s: s = sqrt( ( sum ( ( x - m ) ^ 2) ) / n ); where; x = occurrence (0 or 1)

Prints to STDOUT.

LICENSE

This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.

AVAILABILITY

http://stp.lingfil.uu.se/~evafo/resources/basevocpool/

VERSION HISTORY

Created: 2005-01-10: Eva Forsbom

$Log: dispersion_and_fmod.html,v $

Revision 1.1 2006/08/11 12:31:59 eva

HTML documentation for Perl scripts

Revision 1.3 2006/08/11 12:10:04 eva Added pod usage to Perl files

Revision 1.2 2006/08/08 17:39:03 eva Added pod documentation.