October 2005 — Features
Print this article | Email this articleClick here to receive your FREE subscription to T.H.E. Journal
Scrubbing Data for D3M
Live by the Scrub,
Die by the Scrub
In the end, data scrubbing or cleansing is
the most critical step in the data-collection process toward effective analysis that
ultimately results in top-flight data-
driven decision-making. To get there,
attention must be paid by all participants
in the collection process to the procedures
in place at each school site for the standardization of data entry. And while
scrubbing data is the responsibility of all
individuals involved in the process, ultimate accountability must always reach the
individual who entered the data. To ensure
entry of clean data, a process of validation
must begin with that individual. Experts
recommend a system that returns dirty
data to their source for proper reentry.
When all involved recognize that data
quality is an individual responsibility—
and the magnitude of the potential consequences from analysis and decisions based
upon dirty data are realized—effective
and consistent D3M will be the result.
5 STEPS …TO A SPECIFIC
AND CONSISTENT
PROCEDURE FOR
REQUESTING DATA
• Use a standard form to request data
• Be specific about the information sought
• State the purpose for the request
• Explain how the data are needed
• Inform as to how often the data
will be requested
3TENETS FOR DATA SCRUBBING
• Attention must be paid by all participants
in the collection process to the
procedures in place at each school site
for the standardization of data entry.
• Scrubbing data is the responsibility
of all individuals involved in the
process; however, ultimate accountability
must always reach the individual
who entered the data.
• A validation process must begin with
the individual who entered the data;
return dirty data to the entry source for
proper reentry.
Neil Mercurius is the director of information/education technology and assessment at Jurupa Unified School District (CA).
Cite this Site
copy text (above) for proper citation
